Blog Topics...

3D plotting (1) Academic Life (2) ACE (18) Adaptive Behavior (2) Agglomeration (1) Aggregation Problems (1) Asset Pricing (1) Asymmetric Information (2) Behavioral Economics (1) Breakfast (4) Business Cycles (8) Business Theory (4) China (1) Cities (2) Clustering (1) Collective Intelligence (1) Community Structure (1) Complex Systems (42) Computational Complexity (1) Consumption (1) Contracting (1) Credit constraints (1) Credit Cycles (6) Daydreaming (2) Decision Making (1) Deflation (1) Diffusion (2) Disequilibrium Dynamics (6) DSGE (3) Dynamic Programming (6) Dynamical Systems (9) Econometrics (2) Economic Growth (5) Economic Policy (5) Economic Theory (1) Education (4) Emacs (1) Ergodic Theory (6) Euro Zone (1) Evolutionary Biology (1) EVT (1) Externalities (1) Finance (29) Fitness (6) Game Theory (3) General Equilibrium (8) Geopolitics (1) GitHub (1) Graph of the Day (11) Greatest Hits (1) Healthcare Economics (1) Heterogenous Agent Models (2) Heteroskedasticity (1) HFT (1) Housing Market (2) Income Inequality (2) Inflation (2) Institutions (2) Interesting reading material (2) IPython (1) IS-LM (1) Jerusalem (7) Keynes (1) Kronecker Graphs (3) Krussel-Smith (1) Labor Economics (1) Leverage (2) Liquidity (11) Logistics (6) Lucas Critique (2) Machine Learning (2) Macroeconomics (45) Macroprudential Regulation (1) Mathematics (23) matplotlib (10) Mayavi (1) Micro-foundations (10) Microeconomic of Banking (1) Modeling (8) Monetary Policy (4) Mountaineering (9) MSD (1) My Daily Show (3) NASA (1) Networks (46) Non-parametric Estimation (5) NumPy (2) Old Jaffa (9) Online Gaming (1) Optimal Growth (1) Oxford (4) Pakistan (1) Pandas (8) Penn World Tables (1) Physics (2) Pigouvian taxes (1) Politics (6) Power Laws (10) Prediction Markets (1) Prices (3) Prisoner's Dilemma (2) Producer Theory (2) Python (29) Quant (4) Quote of the Day (21) Ramsey model (1) Rational Expectations (1) RBC Models (2) Research Agenda (36) Santa Fe (6) SciPy (1) Shakshuka (1) Shiller (1) Social Dynamics (1) St. Andrews (1) Statistics (1) Stocks (2) Sugarscape (2) Summer Plans (2) Systemic Risk (13) Teaching (16) Theory of the Firm (4) Trade (4) Travel (3) Unemployment (9) Value iteration (2) Visualizations (1) wbdata (2) Web 2.0 (1) Yale (1)

Monday, July 26, 2010

Overfitted Models and Overly Complicated Regulation...

Governments are always legislating the last crisis (whether economic or security related).  I would argue that one of the reasons that they persist in doing this is that they always seem to craft overly complicated and excessively precise legislation.  This legislation seems to focus on patching specific "holes" in the existing regulation regime that were identified ex post as that the "causes" of the crisis.

As a result of the high levels of complexity and precision, such legislation is likely to be brittle/fragile and will likely result in unintended consequences.  In a sense, such legislation suffers from problems similar to those of an over-fitted statistical model.  A good model will capture key drivers of the underlying process while allowing for some (perhaps even significant variation) between the estimated and observed values.   Because it has captured key drivers of the underlying process, a good model will be very robust when used to predict future values.  An over-fitted model, on the other hand, will be crafted to fit the historical data with very high precision, but because of this high level of historical precision the model can be very brittle when used to predict future values.  Instead of capturing key drivers of the underlying process, an over-fitted model simply reproduces the historical data with high accuracy...who knows what it might produce when used to predict future values.

How does this relate to government legislation/regulation? Good legislation/regulation should capture in broad strokes the key factors associated with a crisis or socio-economic process that the government legislation is designed to target.  Now clearly this is easier said than done as what actually constitutes a "key factor" in the real world of politics is just as debatable as what constitutes a "key driver" in the real world of economic modeling. 

Nonetheless, I think it would be useful if more people new the difference between an over-fitted model and a good model...      

1 comment:

  1. I wonder if this applies to something like the Glass-Steagall act in the USA. It was, by today's standards, short and sweet. Traditional commercial banks, which offered services such as mortgages, checking/savings accounts, could not be involved with investment banking. I presume that was because all commercial bank services were either explicitly or implicitly guaranteed by the Government. It was a simple and robust model. Derivatives, securitizations, etc, belong to the realm of the investment bank. The answer to the financial crisis has been an over-fitted complex model that hopes to give clear indication of systemic risk or risk to a particular bank. Based on your writings, if I understand them, this is a joke.