Learning path

Full curriculum

Full curriculum

Unit content

Weighted least squares with measurement uncertainties

Ordinary least squares treats all residuals as equally informative. In experiments, measurements often have different known standard uncertainties $\sigma_i$. Then a residual of $0.1$ may be negligible for a noisy point with $\sigma=1$ but serious for a precise point with $\sigma=0.01$.

For independent approximately Gaussian measurement errors, weighted least squares minimizes $$\chi^2(\theta)=\sum_{i=1}^n\frac{[y_i-f(x_i;\theta)]^2}{\sigma_i^2}.$$ Equivalently, each point receives weight $w_i=1/\sigma_i^2$. More precise observations influence the fit more strongly because their deviations are less plausibly explained by measurement noise.

Consider estimating a constant $\mu$ from two measurements: $10.0\pm1.0$ and $12.0\pm0.5$. The weighted estimate is $$\hat\mu=\frac{10/1^2+12/0.5^2}{1/1^2+1/0.5^2} =\frac{10+48}{1+4}=11.6.$$ The second measurement pulls the result closer to $12$ because it has four times the weight.

If measurement errors are correlated, a covariance matrix $\Sigma$ replaces independent weights: $$\chi^2=(\mathbf y-\mathbf f)^{\mathsf T}\Sigma^{-1}(\mathbf y-\mathbf f).$$ Ignoring correlation can make a fit appear to contain more independent information than it actually does.

Weights should represent the measurement model, not be chosen afterward to make a curve look better. If the quoted uncertainties themselves depend on fitted parameters or the noise is strongly non-Gaussian, a likelihood model may be more appropriate. Weighted least squares is therefore best understood as a statistical consequence of a particular uncertainty model, not as a cosmetic fitting option.