Theory of Value: Final Thoughts...

I have just finished reading the final chapter on uncertainty in Debreu's Theory of Value.  This final chapter which is very short, simply introduces the idea of contingent commodities and then sketches how the theorems and proofs in previous chapters go through in this more general case.

Instead of sharing my thoughts about contingent commodities, I thought I would post some of my over all thoughts about the book and about general equilibrium more generally...

After reading this book I feel like I have a much improved understanding of both the mathematics and the economics of general equilibrium theory.  For an aspiring academic economist, this is clearly a good thing.  Unfortunately, I do not feel like I have a better understanding of how real-world  economies generally behave.   This is clearly not a good thing. 

After reading, say Minsky's Stabilizing and Unstable Economy, I felt that I was better equipped to talk about issues of serious importance in modern, capitalist economies.  I do not feel that way after reading Theory of Value.  General equilibrium, in my view at least, is not supposed to describe real-world economies but instead serves as a kind of null model for how an idealized economy should behave.  Thus perhaps comparing Theory of Value and Stabilizing and Unstable Economy is not very fair. 

However, even as a null model of the economy, I think general equilibrium falls short.  Any null model of the economy, in my opinion, must allow for the possibility that individual optimization decisions are influenced by the decisions of other individuals in the economy.  In the real world, economic behavior is a very social activity and preferences and individual decisions are heavily influenced by others actions and beliefs.  In the real world, there is also lots of trades.  Perhaps even lots of "false trades" (by false trades I mean trades at non-equilibrium prices).  In GE no one trades until the equilibrium price vector has been calculated, and then they only trade once. 

Once one allows for the possibility that a single agent's decisions can impact the decisions (and/or influence the preferences) of other agents, then micro-dynamics may no longer average-out in the aggregate.  Once one allows for "false trades" each false trade alters the wealth distribution amongst the agents in the economy which then shifts the equilibrium to which the economy would have converged.  In this world "equilibrium" is a moving target.

None of the above critiques of GE are original, and I had encountered all of them prior to reading Theory of Value.  Despite my criticisms, I am still very glad that I read the book and would recommend it to anyone who plans on pursuing an academic career in economics... 

Theory of Value, Chapter 6...

Just finished reading Chapter 6: Optimum.  Debreu lays out the conditions necessary for the equilibrium concept described in the previous chapter to be an optimum for the economy, and under what conditions any optimum can be supported as an equilibrium of a private ownership economy. 

Convexity of individual consumer sets, and of the overall economy's production possibilities set plays is important role for proofs to obtain.  Although the convexity requirement on the production possibilities set is only required to prove that any optimum can be supported by a market equilibrium under a certain price vector.

Only other comment is to point out that, according to Debreu, optimums are in general not comparable to one another (except in trivial case where everyone is simply indifferent between the two optimums).  Although, it is not entirely clear to me how this squares with the idea of Pareto dominance which is sometimes used to eliminate some market equilibria.  To compare between two optimums would require, I think, being able to make inter-personal comparisons of utility...    

Theory of Value, Chapter 5...

Chapter 5 of Debreu's Theory of Value is on economic equilibrium.  First some definitions...let xi be the vector of consumptions of for consumer i with i indexed from 1,...,m; yj be the vector of production for producer j with j indexed from 1,...,n; let wi denote the resource endowment of consumer i (the sum of these endowments over 1,...,m equals w or the total resources of the economy); finally, sij is consumer i's share in the profits of firm j. Net demand is x-y, excess demand is defined to be x-y-w.

The first few sections in the chapter define what an economy is, what market equilibrium is, and what attainable states of an economy are.  Basically attainable states of the economy are states where consumers are choosing a consumption bundle that is possible for them (i.e., satisfies their wealth constraint), producers are choosing a production that is possible for them, and that the market is in equilibrium (i.e., the net demand must equal the total available resources).  Economy equilibrium is then defined to be some subset of these obtainable states where consumers are maximizing utility and firms are maximizing profit.

Some questions about private ownership...
Private ownership of the means of production, I think, is simply Debreu's way of allocating the pure profit that producers make in equilibrium when production processes exhibit decreasing returns to scale.  How are the shares determined?   This does not seem to be addressed.  If all firms are identical, then it doesn't matter.  However if firms are not identical, then it seems to me that consumer i's wealth (and by extension his choice of consumption) would change depending on his idiosyncratic portfolio of shares in the producers.  There seems to be a missing market for stocks in this world... 

On an unrelated note: I have always felt that the shape of the production possibilities set was technologically determined, and that it was shares in this technology that where owned by the consumers (who I suppose may or may not also be the workers).  These shares then give a consumer claims to the proceeds from the sale of the output produced.  The point is, I have always thought of the consumers owning the production technology itself and not simply owning the output of production.  Anyone have thoughts on this? Do consumers own the technology of production? Or do they simply own the output?  If it is the latter, then who owns the technology of production? 

Proof of Existence: I will not comment on the proof, except to say that Debreu proves only existence and does not show uniqueness of stability.  

Also...in the end of chapter notes Debreu cites a paper by L.W. McKenzie called "Competitive Equilibrium with Dependent Consumer Preferences" which looks really interesting...unfortunately I can not seem to find in cursory Google search...

Theory of Value: Chapter 4...

Having just finished putting the finishing touches on my lecture slides for my talk at tomorrow's workshop of network fragility here in Edinburgh, I am back to reading Gerard Debreu's Theory of Value. I am currently in the chapter on consumer theory.

I like the way Debreu emphasizes that the indifference relation is a complete binary relation that partitions the consumers choice set (i.e., the indifference relation is reflexive, symmetric, and transitive, and complete).  The interest in the utility function then follows from the fact that we would like to have some increasing function that associates each indifference class with a real number that can be used to distinguish it from other indifference classes.

The proof of existence of a utility function when one assumes a form of continuity of preferences is quite clever.  The proof shows that there exists a dense subset of a consumer's choice set.  Defines a clever increasing function on that subset, and extends the function from the dense subset to the entire choice set.  Then the function is shown to be continuous.

As in producer theory, convexity of the choice set is crucial.  Working through the three different types of convexity: weak-convexity, convexity, and strong convexity was worthwhile.  Weak-convexity allows "thick" indifference curves, convexity rules out such "thick" indifference curves, strong-convexity is the type of convexity that is taught to 1st year undergraduates as being one of the reasons that marginal rates of substitution decrease as one moves down an indifference curve. 

The wealth constraint. The proof of existence of equilibrium in a private ownership economy rests crucially on the continuity of the correspondence between the set of price-wealth pairs such that the set of possible consumption bundles is not empty and the choice set of our agent.  Why? I will let you know after I have read chapter 5 on equilibrium.  For now I cite Debreu...(I am going to guess that continuity of the correspondence is necessary to insure that our profit maximizing producers do not choose to produce an amount of output that falls into a "hole" so to speak in the set of consumer utility maximizing bundles...I will let you know if this intuition turns out to be correct or not!)   

Theory of Value: Chapter 3 (cont'd...Again!)...

Last post on producer theory.  In his end of chapter notes, Debreu makes underlines three things that are not covered by the producer theory that he has described.  I repeat them below as I think they merit attention:

1. External economies and diseconomies: the case where the production set of a producer depends on the production sets of the other producers (and or on the consumptions of consumers).  Both of these cases are, I think, incredibly likely to occur in the real-world.  In fact such interdependencies are well modeled by networks.
2. Increasing returns to scale: one of my new favorite pastimes...these can also be well modeled with networks (although they can be well-modeled via other methods as well).
3. The behavior of producers who do not take prices as given: monopolistic competition...I would go so far as to submit that some form of monopolistic competition is a superior null model of producer behavior than perfect competition.

Theory of Value: Chapter 3 (cont'd)...

Producer Theory and Profit Maximization...where exactly are opportunity costs accounted for within the general equilibrium framework?  This question was posed to me by one of my first year undergraduates this year (in a slightly different form!) and I don't think I had a very good answer for him.

As I read through Debreu's axiomatic treatment of profit maximization I find myself asking the same question.  Where are opportunity costs taken into account?  Are opportunity costs essentially a special type of contingent commodity that exists in perhaps a different time and place with its own price?

This matters because the assumption of additivity and the possibility of inaction implies that the maximum profit of a producer either does not exist or is null.  Null profit in equilibrium makes sense to me IF one is talking about economic profits and not accounting profits.  Economic profits requires taking opportunity costs into account...

Anyone out there have any thoughts on this one...or is this discussion just too pedantic 

Theory of Value: Chapter 3...

Chapter 3 is on producer theory.  I was rolling right along without problems through the first few pages until I encountered the following:

"A production yi is classified as possible or impossible for the ith producer on the basis of his present knowledge about his present and future technology.  The certainty assumption implies  that he knows now what input-output combinations will be possible in the future (although he may not know the details of the technological process which will make them possible)."

How could you know the input-output combinations that are possible in the future without knowing the technology?  Seems a bit weird to assume certainty, but then to also assume that producers have perfect knowledge about everything except the technology used to produce things. 

A more interesting comment appears in Debreu's discussion of the various assumptions made on a producer's production possibilities set.  While discussing various interpretations of the additivity assumption, Debreu writes as follows:

"In so far as the [production possibilities set] for a producer represents technological knowledge, it is clear that two production plans separately possible are jointly possible.  Alternatively the jth producer can be interpreted as an industry rather than a firm; then the additivity assumption means that there is free entry for firms into that industry.  Under additivity if yj is possible than so is kyj, where k is any positive integer.  Therefore additivity implies a certain kind of non-decreasing (i.e., increasing or constant) returns to scale."

It is this last comment that additivity implies a certain kind of non-decreasing returns to scale that stopped me.  I see why k has to be an integer (additivity implies that yj + yj +...+yj  = kyj must also be possible). I suppose I had just forgotten that constant returns to scale act as lower bound when we assume additivity (i.e., that decreasing returns to scale are not possible).

The next assumption discussed is convexity.  Convexity implies non-increasing returns to scale (convexity plus the no-free-lunch assumption rules out increasing returns).  Thus if one wants to assume additivity and convexity of the production set for a particular producer, then the production technology must exhibit constant returns to scale.    

Theory of Value, Chapter 2...

So, as yet another side project, I am reading Gerard Debreu's Theory of Value: An Axiomatic Analysis of Economic Equilibrium.  It has rekindle my interest in abstract mathematics, and as an economist it has so far proved helpful in understanding the particularities of General Equilibrium theory in more detail.

Right now I am reading Chapter 2: Commodities and Prices.  From my MSc I was aware that Arrow-Debreu general equilibrium assumed the existence of markets for all commodities, where commodities are completely specified by their intrinsic characteristics, the time that they are acquired, and their location (i.e., Red Winter Wheat, today, in Chicago is a different good from Red Winter Wheat, a year from now, in San Francisco, etc.).

I was not aware however, that the commodity space that defines all possible combinations of these commodities has finite dimension and that time is also taken to be finite.  Even the claim that the commodity space has finite dimension for a fixed moment in time seems to be implausible.  Intuitively, economic growth would seem to require (or be driven by) continual innovation of new commodities, but I am also not sure how one is to think of commodities that have not been created yet with this framework.  Are they to be accommodated by allowing the dimensionality of the commodity space to increase with time?  Or perhaps this is being abstracted from in the general equilibrium framework.

Debreu addresses some of these critiques in his end of chapter notes.  Note 2 says that it is the assumption of finite time that allows the commodity space to be of finite dimension.  He goes on to say that many of the results to follow can be extended to an infinite dimension commodity space.  I am still not sure whether this addresses my concern about the ability of the theory to deal with commodities that have yet to be invented...

I should mention that I do find the theory quite elegant.  It kind of cool the way you derive the exchange rates, interest rates, and discount rates from the price system as long as you have a unit of exchange.  Here it is assumed that there exists some unit of exchange (I suppose that this is why so many economists have devoted their careers to developing theories of where money comes from...which is something else that I have never understood!) 

A Great Loss...

Benoit Mandelbrot died today...a great light has gone out...Such an intellect comes only a few times each generation.  He was a truly original thinker, and one of my role models...

He will be truly missed.

Introductory Maths and Stats: Intertemporal Optimization...

This lecture should be cut.  Material covered is not really used enough in the core curriculum to justify spending anytime on this...some version of the material could be included as a separate handout over winter holiday.

Introductory Maths and Stats: Discrete-time Intertemporal Optimization...

This should be the culminating lecture of QM0.  Students should be able to understand the difference between static and dynamic optimization.  The intuition for this can be built by focusing time on explaining how one derives the life-time budget constraint from the budget constraints for each time period (basically just add/aggregate, integrate depending).  Really invest time in going through the details of exactly what the budget constraint is how it is derived etc.  This is important as it comes up again and again...

Intertemporal Choice: Two-Period Example: Wouldn't really change much from this section, it is nicely written and hits all the high points...

Intertemporal Choice: T-Period Example: Here the focus is on the permanent income hypothesis, which is a pretty good example that demonstrates the techniques involved.

More Complicated T-period Example: This section should be cut from the lecture and covered in tutorials...this would allow lecturer to move through the above material at more measured pace.  I would jump from the permanent income hypothesis material straight to the simple discussion of dynamic programming.

Simple Discussion of Dynamic Programming: Section is good, although the maths needs to be simplified a bit so as to coincide with the permanent income hypothesis section that would proceed it.  Perhaps tutors could extend the dynamic programming case in the tutorials...

Introductory Maths and Stats: Kuhn-Tucker Theorm...

Intuition: Since most economic constraints are inequality constraints not equality constraints, it makes sense for students to learn a bit about Kuhn-Tucker theory...to build intuition I like to draw pictures in order to demonstrate the different sets of complementary slackness conditions for a single variable function in both the maximization case and the minimization case.  There would be six diagrams that clearly emphasize the corner solutions v. the interior optimum, and the idea of a binding constraint versus a slack constraint.  Remember, if one of the constraints is slack the other MUST be binding!

Kuhn-Tucker Theorem: After building intuition with diagrams in the single variable case, I would jump straight to the Kuhn-Tucker theorem and the corresponding algorithm used to solve inequality constrained optimization problems.

General Case: The notes on the general case are confusing and I am not sure that they add to the student's understanding of how to apply Kuhn-Tucker.  I would recommend cutting the notes on the general case and spend more time working problems and making sure that the students understand the difference between slack constraints and binding constraints...

Introductory Maths and Stats: Static Unconstrained Optimization of N-Variable Function...

The title is a mouthful, but the lecture itself is fairly straightforward (aside from the notation being a bit complex)...

Rules for Single Variable Optimization:

• If df/dx=0 and d^2f/dx^2<0 at any point x0, then x0 is a local max
• If df/dx=0 and d^2f/dx^2>0 at any point x0, then x0 is a local min
• If df/dx=0 and d^2f/dx^2=0 at any point x0, then necessary but not sufficient conditions exist for x0 to be an inflexion point...

N-variable case is similar, but the notation becomes more complex (uses vectors instead of single variables etc.) 

The Two Variable Case: The discussion in the lecture notes of unconstrained optimization with two variables is good.  I particularly like how emphasis is placed on using Taylor expansions in the argument.  I would only recommend that more pictures be included.  Anytime a Taylor expansion is used, it just screams DRAW A PICTURE!!!

Quadratic Form, Definite Matrices and Hessians: I would like to see this discussion moved up a bit.  Hessians should be introduced in lecture 2 on multi-variable calculus.  The maths notes should link more closely with the stats notes (particularly the linear algebra parts).   Definite matrices should be emphasized in both the maths and stats, and a solid amount of lecture and tutorial time should be spent on the concept.  Definite matrices provide the coat-hanger on which much of the linear algebra that is used in microeconomics and QM hangs...

Concavity and Convexity: Again draws pictures.  Emphasize that the definitions are almost identical to the single variate case.  Only difference is that we are dealing with vectors now and not scalars in the argument of the function.

Chain Rule and the Envelope Theorem:  Material on the Envelope Theorem is scattered across three lectures.  I think the best think to do is devote an entire lecture to the envelope theorem after all of the necessary maths have been developed.  This would serve as a useful mid-course refresher for the students, and I think would make the theorem more understandable.  It is important, and thus I think it should get its own lecture...

Economic Applications: Solow Efficiency Wage model should be cut out of lecture and covered in a tutorial.  This would open up more lecture time for other more important topics...

The entire section that covers the derivations of the OLS equations using maximum likelihood should be cut from the lecture and converted into a handout for the students to study over winter holiday, it is very long and too complicated to ask about on the QM0 exam.  Lecture time and tutorials would be better spent elsewhere... 

Introductory Maths and Stats: Multi-Variable Calculus...

This is a continuation of my notes for my intro maths and stats tutorials.  This is my summary of Lecture Two: Multi-Variable Calculus...

Partial Differentiation: Easy to extend differentiation from single variable to multi-variable case.  Say you have f(x,y), then to take partial derivative with respect to x simply treat y as a constant and take the derivative of f with respect to x like single variable case!  That's it...also higher order derivatives are calculated by successive application of differentiation.  Demonstrate that cross-partial derivatives are equal (if f is well-behaved)!

I think it would be worthwhile to also mention the Hessian Matrix (matrix of second derivatives).  Talk about special cases when matrix is positive (semi) definite of negative (semi) definite.  Can also use it as an excuse to talk about eigenvalues, eigenvectors, determinants, etc from linear algebra.  Example: f(x,y)=x^2 + y^2...

Total Differentiation and Chain Rules: I totally agree with Yu Fu...one should not try to memorize all of the chain rules related to partial differentiation there are just too many combinations and cases.  Better to focus on understanding the concept of total differentiation and then the difference between independent and intermediate variables.  For example: suppose we have the usual case in economics where f(x(t), y(t)) and t=time.  In this case the independent variable is t, and the dependent variable is f (x and y are only intermediate variables that "filter" the effect of t on f). 

Implicit Functions and Differentiation: Just another application of partial differentiation and chain rules...

The Envelope Theorem: Understanding the envelope theorem is key in microeconomic price theory.  Mathematically, the envelope theorem is simply an application of chain rules, total differentiation, and partial differentiation!  No sweat...

Systems of Implicit Functions and Jacobian Determinants: BLAH!  OK, first I think the lecture notes need to be re-ordered so that the lecturer reviews determinants, Cramer's rule etc. BEFORE tackling this section.  Note that Cramer's rule is a REALLY inefficient way to solve a system of linear equations!  For QM0 exam the students may have to compute 3x3 determinant, so they need to know a formula for it...I would go with the Co-factor expansion...

Leibnitz's Rule: This is a cut I think...should be covered in detail by lecturer on Ramsey model in Macroeconomics I...

Integration with Several Variables: Move towards the beginning...this is very straightforward and should probably be talked about right after partial differentiation...

Homogeneous and Homothetic Functions: This is a cut.  Not because it isn't important...it is very important (implications of CRTS and such) but I think that the lecturer should cover these topics in class during term.  There is already too much material in the QM0 lecturers and this would allow for more detailed coverage of other topics...

Linear Dynamic System: If we want to keep this material in course, then we need to do a much better job of teaching eigenvalues, eigenvectors, and matrix diagonalization techniques.  Would recommend moving Appendix on eigenvalues, and eigenvectors into the lecture notes and teaching students how to reach the general solution of a linear dynamic system properly...

Introductory Maths and Stats: Single Variable Calculus...

As a first year Phd student I will be teaching introductory maths and stats to the MSc students this year. I am going through the lecture notes and making little notes for myself about things that I think should be emphasized (or de-emphaszied) in the turorials as well as some little tricks that I have picked up along the way that should be helpful for the incoming MSc students.  The following are my notes to myself on single variable calculus...

Rules of Differentiation: The derivtive is a linear operator.  Mathematically this means that d/dx(f(x) + g(x))=d(f(x)) + d(g(x)) and d/dx(t*f(x))=t*d/dx(f(x)).  In words this means that the derivative of any linear combination of well-behaved functions is equal to the same linear combination of the derivatives of the individual functions.  Note that if you remember that the derivative is a linear operator then you automatically know how to take derivatives of sums and differences of functions.

Other Rules I remember:

1. Constant: The derivative of a constant is always zero.
2. Powers: If f(x)=x^k, then df(x)/dx=kx^(k-1)
3. The Chain Rule: NEVER forget the chain rule! d/dx(f(g(x)))=df/dg*dg/dx.  Most simple mistakes in taking a derivative come from forgetting about the chain rule.
4. Derivative of the exponential and the natural logarithm functions: Easy...d/dx(e^x)=e^x (this result is one of the reasons that exponential functions turn up so often in the general solutions to differential equations), and d/dx(ln(x)=1/x.  Maybe review some basic properties of logarithms and exponentials...
5. Product Rule: d/dx(f(x)*g(x))=d/dx(f(x))*g(x) + f(x)*d/dx(g(x))

Rules I never remember:

1. Quotient Rule: Why? Because the quotient rule is simply an application of the product rule and the chain rule.
2. Rule for d/dx(a^x): Why? Because it is better to just take natural logarithms and the differentiate.  For example, if f(x)=a^x then ln(f(x))=ln(a^x) and because I know the my properties of logarithms, the rule for taking d/dx(ln(x)) and the chain rule this becomes d/dx(ln(f(x))=[1/f(x)]*d/dx(f(x))=d/dx(ln(a^x))=ln(a) and finally d/dx(f(x))=ln(a)*a^x

l'Hopital's Rule: Unbelievably useful for taking limits.  Basically if you are ever in the case where the lim of two functions turns out to be 0/0 or inf/inf, then take deriviates of the top and bottom and take limits again...

At this point the lecture notes have a discussion of 1st order differential equations that is out of place.  These equations have not been covered yet in the notes, and even though this discussion is brief it detracts from more important material.  Lecture notes also have a long digression on stock returns, capital gains, and dividends.  This is an important economic application of the material being taught, but should be covered by tutors in the QM0 tutorials where it can be gone over at a slower pace...

Optimization: Recall geometric interpretation of derivative: the value of a derivative at a given point tells you whether the function is increasing of decreasing at the point:

• If df/dx>0, then the function is increasing
• If df/dx<0, then the function is decreasing
• If df/dx=0, then f has a critical point

At this point I like to draw pictures to help me remember that d^2f(x)/dx^2>0 (<0) implies that the function is convex (concave). Which leads to the corresponding definitions of maximums and minimums of functions.  I like to point out the main ideas both graphically and in terms of the FOC and SOC on derivatives.

Taylor Expansions: Important topic that the lecturer should spend more time laying out the details.  Re-empahsis should be placed on the Taylor Expansion in the tutorials.

Concavity/Convexity and Quasi-concavity/Quasi-convexity of Functions: I never remember the  derivative or algebraic definitions for these terms.  Best to draw pictures!  Three functions to remember f=x^2 (convex), f=ln(x) (concave), and f=x^3 (quasi-convex and quasi-concave)

Rules of Integration: Emphasize the area under the curve interpretation of an integral.  The rules for integration are easy IF you know your rules for differentiation.  The two processes work in reverse.  When taking an integral of f(x), think what function would I need to take the derivative of the get the function f(x).  Don't forget about the arbitrary constant!

I Under-appreciated the Kronecker Product...

Today, I came to the realization that I massively under-appreciated the Kronecker product of two matrices.  Until today, I thought that it wasn't all that useful...boy was I wrong!  Kronecker products are wonderfully useful, particularly if you are interesting in networks.  Here are links to a some papers on using Kronecker products as a method for generating network graphs that have power law degree distributions and small diameter.  I will write additional posts on this topic later, as I expect that it will be extremely useful in my own research...

Online Lecture Series on Differential Equations...

MIT OCW saves the day again!  A whole course on solving ODE's from MIT's Arthur Mattuck...

Another Brilliant Linear Algebra Lecture...

Prof. Gilbert Strang at MIT delivers a beautiful lecture on how to solve systems of first order differential equations using linear algebra...an absolute must for anyone studying graduate economics...

First Section of Lecture Notes on Geometry of Linear Regression...

This is my first cut of lecture notes on the Geometry of Linear Regression...FYI the b and beta are the same...having html issues.  Hopefully I have not made any egregious errors...

The Geometry of Linear Regression

Suppose we have the following system of equations:

y=Xb

Here the dependent variable y is a vector of length m, X is our (m x n) matrix (i.e., m rows and n columns, typically m>n) of independent variables, b is a vector of coefficients of length n.  Why are we going to start by talking about the geometry of solutions to systems of linear equations? Well, because at a fundamental level linear regression is really all about "solving" a system of linear equations when there is no true solution.  Linear regression finds a solution b to our system of equations that is the "best" because it is "closest" in a very specific way to the vector y.

Now our system of m equations with n unknowns (the n coefficients which comprise the vector b) tells us that the vector y (our dependent variable) is a linear combination of the columns of X (our independent variables)….

y= b₁x₁ + b₂x₂  + … + b_nx_n

Here x_i i=1,…n are the column vectors of length m that make up the matrix X.  This means that the vector y is in the column space, col(X), of our matrix X. In pictures with 2 independent variables…notice that the our independent variable, the vector y, lies in the plane corresponding to the col(X)

Remember from its definition that the col(X) is the vector space spanned by the column vectors of X, which is simply a fancy way of saying that the col(X) includes all linear combinations of the column vectors of  X (which includes y at this point).  If the column vectors, our dependent variables, also happen to be linearly independent of one another then our column vectors form a basis for the col(X).  Normally this will be the case…but it is crucial that our set of dependent variables be independent of one another!  

If we have nice case: X is an (m x n) matrix with m>n and that our columns of X, which span the col(X) by definition, are linearly independent of one another and thus also form a basis for the col(X).  This implies that the rank of X (which as you will remember is simply the number of linearly independent columns of X) and the dimension of col(X) (which is simply the number of vectors needed to form the basis of col(X)) are both equal to n.  Our matrix has full column rank! We are off to a good start…

Let’s Talk About Correlation…

Geometrically, correlation between two variables (which we are representing as vectors) is related to the angle between two variables/vectors via the following formula…

Cosine! Theta! Dot products and Euclidian Norms! Boo! Let’s draw pictures…In this first picture our two independent variables are positively (negatively) correlated because the angle between their two corresponding vectors in the col(X) is acute (obtuse).  I draw the positively correlated case below… 

In this second picture, the two vectors are at right angles with one another and are therefore uncorrelated.  This is an extremely important case…when you are learn about OLS, IV and GLS the question of whether or not your error term is uncorrelated with your explanatory (i.e., independent) variables will come up again and again…remember, geometrically, uncorrelated mean vectors at right angles!

Finally what does it look like if the two vectors are perfectly positively (negatively) correlated with one another?  Although I will leave it up to you to draw your own picture, for the perfectly positively correlated case look at the picture of the acute case and think about what happens as the angle gets really, really small.  Once you figure that out and get your picture, the perfectly negatively correlated case is simply the 180-degree (hint) opposite…

To be continued in the near future with a simple linear model y=Xb + e!

Still Watching Linear Algebra...

For those interested in an introduction to networks, graph theory, and linear algebra lectures 11 and 12 from Gilbert Strang at MIT are brilliant...