As I am prepping for my teaching requirements for the upcoming year, I have been thinking about employing a more visual/geometric approach to OLS. I remember sitting in a lecture on GMM where the professor was talking about projection matrices and all of the sudden something just clicked. I recall thinking: "Wow, the different versions of regression analysis (i.e., OLS, GLS, IV, etc) can be thought of as projections of the dependent variable (y) onto the column space spanned by the regressors (Xs)." When you multiply the vector y by the projection matrix P = X(X′X)−1X′ you project the vector y onto the space spanned by the columns of X and the result is shown in the diagram below.
The matrix Xβ is simply a linear combination of the columns of X (i.e., a linear
combination of the regressors) and as such lies in the column space of X. The idea
behind OLS is to then simply choose the right β so that the linear combination will
minimize the distance between the vector y and its projection in the column space of X.
Additionally it is straightforward to extend this framework to talk about issues related
to correlation between regressors, errors, etc.
combination of the regressors) and as such lies in the column space of X. The idea
behind OLS is to then simply choose the right β so that the linear combination will
minimize the distance between the vector y and its projection in the column space of X.
Additionally it is straightforward to extend this framework to talk about issues related
to correlation between regressors, errors, etc.
I would like to emphasize this projection interpretation to my introductory statistics and econometrics students. Although this approach does use some basic jargon borrowed from linear algebra (most of which they are already required to know)...I think/hope that it will be more intuitive for the students.
No comments:
Post a Comment