That’s my question, I have looking round online and people post a formula by they don’t explain the formula. Could anyone please give me a hand with that ? cheers
If the OLS regression contains a constant term, i.e. if in the regressor matrix there is a regressor of a series of ones, then the sum of residuals is exactly equal to zero, as a matter of algebra.
For the simple regression,
specify the regression model
Then the OLS estimator (ˆa,ˆb) minimizes the sum of squared residuals, i.e.
For the OLS estimator to be the argmin of the objective function, it must be the case as a necessary condition, that the first partial derivatives with respect to a and b, evaluated at (ˆa,ˆb) equal zero. For our result, we need only consider the partial w.r.t. a:
But yi−ˆa−ˆbxi=ˆui, i.e. is equal to the residual, so we have that
The above also implies that if the regression specification does not include a constant term, then the sum of residuals will not, in general, be zero.
For the multiple regression,
let X be the n×k matrix containing the regressors, ˆu the residual vector and y the dependent variable vector. Let M=In−X(X′X)−1X′ be the “residual-maker” matrix, called thus because we have
It is easily verified that MX=0. Also M is idempotent and symmetric.
Now, let i be a column vector of ones. Then the sum of residuals is
So we need the regressor matrix to contain a series of ones, so that we get \mathbf M\mathbf i = \mathbf 0.