Normal Equations Example

We wish to fit a parabola f(x) = w(1)*x.^2 + w(2)*x + w(3) to the dataset {x_i,y_i} given by x = [2 4 5 7 9 10]', y = [4.4 15.9 20.7 22.3 15.6 11.5]'. Set X = [x.^2 x ones(6,1)], A = X'*X and b = X'*y. Note that cond(X) = 324.3418 (big), whereas cond(A) = 1.0520e+05 (huge) -- this is why nobody uses normal equations for least squares fitting in practice. We then have A = [19859 2269 275; 2269 275 37; 275 37 6], b = [4295.8 587.4 90.4]' and w = A \ b = [-0.8933 11.5718 -15.3538]', i.e. according to normal equations, the best (in the least squares sense) parabolic fit to our data points is f(x) = -0.8933*x.^2 + 11.5718*x - 15.3538. To see what this fitted curve looks like, use pts = 2:.1:10; plot(x,y,'o',pts,-0.8933*pts.^2 + 11.5718*pts - 15.3538). If we use MATLAB's internal least squares solver instead, we get z = X \ y = [-0.8905 11.5374 15.2663]. The difference is due to the poor conditioning of normal equations.