Why the sum of residuals equals 0 when we do a sample regression by OLS?

That’s my question, I have looking round online and people post a formula by they don’t explain the formula. Could anyone please give me a hand with that ? cheers

Answer

If the OLS regression contains a constant term, i.e. if in the regressor matrix there is a regressor of a series of ones, then the sum of residuals is exactly equal to zero, as a matter of algebra.

For the simple regression,
specify the regression model
yi=a+bxi+ui,i=1,...,n

Then the OLS estimator (ˆa,ˆb) minimizes the sum of squared residuals, i.e.

(ˆa,ˆb):ni=1(yiˆaˆbxi)2=min

For the OLS estimator to be the argmin of the objective function, it must be the case as a necessary condition, that the first partial derivatives with respect to a and b, evaluated at (ˆa,ˆb) equal zero. For our result, we need only consider the partial w.r.t. a:

ani=1(yiabxi)2|(ˆa,ˆb)=02ni=1(yiˆaˆbxi)=0

But yiˆaˆbxi=ˆui, i.e. is equal to the residual, so we have that

ni=1(yiˆaˆbxi)=ni=1ˆui=0

The above also implies that if the regression specification does not include a constant term, then the sum of residuals will not, in general, be zero.

For the multiple regression,
let X be the n×k matrix containing the regressors, ˆu the residual vector and y the dependent variable vector. Let M=InX(XX)1X be the “residual-maker” matrix, called thus because we have

ˆu=My

It is easily verified that MX=0. Also M is idempotent and symmetric.

Now, let i be a column vector of ones. Then the sum of residuals is

ni=1ˆui=iˆu=iMy=iMy=(Mi)y=0y=0

So we need the regressor matrix to contain a series of ones, so that we get \mathbf M\mathbf i = \mathbf 0.

Attribution
Source : Link , Question Author : Maximilian1988 , Answer Author : Alecos Papadopoulos

Leave a Comment