Some readers might not like the ’’black box” nature of regression routines on calculators...

Some readers might not like the ’’black box” nature of regression routines on calculators and computers and would prefer to understand regression in more depth. Suppose that we wish to ’’fit” a line having equation y = mx + b to a set of data points (x1, y1), …, (xn, yn). Then the points on the line having the same *-coordinates as the given data points are (x1, mx1 + b), …, (xn, mxn + b). The error made at the rth data point is e,- = yi − (mxi +b). If the i th data point lies above the line of best fit, then this error is positive. If the i th data point lies below the line of best fit, then this error is negative. To keep the sum of the errors from canceling, we square each error before summing. Then the sum of the squares of the errors is given by

Note that S is a function of m and b. The idea behind the ’’line of best fit” is to minimize this sum of the squares of the errors. That’s how the process gets the name least squares fut. We want to pick values of m and b that minimize S. To complete this minimization, we’ll need to find critical values using differentiation.

(a) Take the partial derivative of S, defined by equation (1.23), with respect to m and set it equal to zero to obtain

If you don’t know what a partial derivative is, that’s fine; just differentiate with respect to m, holding b constant. Next, take the partial derivative of S with respect to b (hold m constant) and set it equal to zero to obtain

Solve equations (1.24) and (1.25) simultaneously, eliminating b, to show that

Finally, you can use this value of m and equation (1.24) to calculate b.

(b) Compare equation (1.7) in the narrative with equation (1.23) and then make the appropriate changes to equation (1.26) to show that

Use this result to show that

(c) Use equations (1.27) and (1.28) to compute the values of r and C found in Example Plot the data and superimpose the resulting exponential curve of best fit.