Blog Topics...

3D plotting (1) Academic Life (2) ACE (18) Adaptive Behavior (2) Agglomeration (1) Aggregation Problems (1) Asset Pricing (1) Asymmetric Information (2) Behavioral Economics (1) Breakfast (4) Business Cycles (8) Business Theory (4) China (1) Cities (2) Clustering (1) Collective Intelligence (1) Community Structure (1) Complex Systems (42) Computational Complexity (1) Consumption (1) Contracting (1) Credit constraints (1) Credit Cycles (6) Daydreaming (2) Decision Making (1) Deflation (1) Diffusion (2) Disequilibrium Dynamics (6) DSGE (3) Dynamic Programming (6) Dynamical Systems (9) Econometrics (2) Economic Growth (5) Economic Policy (5) Economic Theory (1) Education (4) Emacs (1) Ergodic Theory (6) Euro Zone (1) Evolutionary Biology (1) EVT (1) Externalities (1) Finance (29) Fitness (6) Game Theory (3) General Equilibrium (8) Geopolitics (1) GitHub (1) Graph of the Day (11) Greatest Hits (1) Healthcare Economics (1) Heterogenous Agent Models (2) Heteroskedasticity (1) HFT (1) Housing Market (2) Income Inequality (2) Inflation (2) Institutions (2) Interesting reading material (2) IPython (1) IS-LM (1) Jerusalem (7) Keynes (1) Kronecker Graphs (3) Krussel-Smith (1) Labor Economics (1) Leverage (2) Liquidity (11) Logistics (6) Lucas Critique (2) Machine Learning (2) Macroeconomics (45) Macroprudential Regulation (1) Mathematics (23) matplotlib (10) Mayavi (1) Micro-foundations (10) Microeconomic of Banking (1) Modeling (8) Monetary Policy (4) Mountaineering (9) MSD (1) My Daily Show (3) NASA (1) Networks (46) Non-parametric Estimation (5) NumPy (2) Old Jaffa (9) Online Gaming (1) Optimal Growth (1) Oxford (4) Pakistan (1) Pandas (8) Penn World Tables (1) Physics (2) Pigouvian taxes (1) Politics (6) Power Laws (10) Prediction Markets (1) Prices (3) Prisoner's Dilemma (2) Producer Theory (2) Python (29) Quant (4) Quote of the Day (21) Ramsey model (1) Rational Expectations (1) RBC Models (2) Research Agenda (36) Santa Fe (6) SciPy (1) Shakshuka (1) Shiller (1) Social Dynamics (1) St. Andrews (1) Statistics (1) Stocks (2) Sugarscape (2) Summer Plans (2) Systemic Risk (13) Teaching (16) Theory of the Firm (4) Trade (4) Travel (3) Unemployment (9) Value iteration (2) Visualizations (1) wbdata (2) Web 2.0 (1) Yale (1)
Showing posts with label Power Laws. Show all posts
Showing posts with label Power Laws. Show all posts

Friday, March 16, 2012

Equity returns: where power-laws go to die? Maybe...

A major criticism of previous empirical work assessing support for the power-law as a model for equity returns is that very few (any?) of the studies assess goodness-of-fit of the power-law model.  This post summarizes the results of goodness-of-fit testing for the power-law as a model for equity returns and is a continuation of my previous posts on the power-law as a model for equity returns using data on U.S. equities listed on the Russell 1000 index.

I implement two versions of the KS goodness-of-fit tests suggested in Clauset et al (SIAM, 2009):
  • Parametric version: conceptually and computationally simple; assumes estimated xmin is the "true" power-law threshold and simulates data under the null hypothesis of a power-law with exponent equal to estimated α
  • Non-parametric version: conceptually less straight-forward and computationally very intensive because it makes use of the fact that  xmin is estimated from the data.
Results:
Using the parametric version of the KS goodness-of-fit test I am able to reject the power-law model as plausible (i.e., p-value ≤ 0.10) for 17% of positive tails and 12% of negative tails.  I would not describe these results as overwhelmingly "anti-power-law," but I would remind readers that the parametric version of the goodness-of-fit test sets a lower-bound on support for the power-law (see my discussion of goodness-of-fit results for mutual funds for more details).  Here is a histogram of the p-values for the positive and negative tails of equities in my sample:
The results obtained using the more flexible non-parametric KS test are much less supportive of the power-law model.  I reject the power-law model as plausible for roughly 44% of positive tails and 37% of negative tails.  Here is another histogram of the goodness-of-fit p-values:
Clearly the non-parametric goodness-of-fit results are more damning for the power-law model than the parametric results.  But why? 

Discrepancy could be due to sample size effects.  The average number of tail observations for a given stock in my sample is 313 (positive tail) and 333 (negative tail): using daily data, I simply do not have that many tail observations to work with.  Because of the small sample size I need to rely the extra flexibility of the non-parametric version of the goodness-of-fit to be able to reject the power-law model as plausible.  I suspect that this would not be the case if I had access to the TAQ data set used by Gabaix et al (Nature, 2003).  It is also worth noting that the more data I get, the more likely I am to reject the power-law as plausible.  Specifically, for those equities for which I reject the power-law model as plausible, either for positive or negative tail (or both), I have a larger average number of both total observations and observations in the tail.  This sample size dynamic has been something I have observed fairly consistently whilst fitting and testing power-law models to various data sets.  The more data I obtain, the less support I find for a power-law and the more support I tend to find for heavy-tailed alternatives (particularly the log-normal). 

While I interpret the above results of goodness-of-fit tests as being decidedly against the hypothesis that the power-law model is "universal," clearly the power-law is a plausible model for either the positive or negative (or both) tails of returns for some stocks.  In my mind this immediately suggests that there might be meaningful heterogeneity in the tail behavior of asset returns and that an interesting research direction might be to explore economic mechanisms that could generate such diversity of tail behavior. 

I end with a significant disclaimer.  I am concerned that by ignoring the underlying time dependence (think "clustered volatility" and mean-reversion) in large returns that my goodness-of-fit test results might be biased against the power-law model.  Suppose that Gabaix et al (Nature, 2003) are correct about equity returns having power-law tails.  Given the dependence structure of returns, it could be the case that the typical KS distance between the best-fit power-law model and the "true" power-law is larger than what I estimate in implementing an iid goodness-of-fit test. 

If this is the case, then perhaps the reason I reject the power-law model so often is because my observed KS distance is obtained from fitting a power-law model to dependent data, whereas my bootstrap KS distances are obtained by fitting a power-law to synthetic data that follows a "true" power-law but ignores the underlying dependence structure of returns.  In order to address this issue, I will need to develop an alternative goodness-of-fit testing procedure that can mimic the time dependence in the returns data!  Fortunately, I have some ideas (good ones I hope!) on how to proceed...      

Monday, March 12, 2012

Zipf's Law does not hold for mutual funds!

Gabaix et al (Nature, 2003) and Gabaix et al (QJE, 2006) lay out an economic theory of large fluctuations in share prices based, in part, on the assumption that the size (as measured in dollars of assets under management) of investors in asset markets is well approximated by Zipf's law (i..e., a power-law with scaling exponent ζ ≈ 1 or α ≈ 2).  Zipf's law has been purported to hold for cities ( Zipf (Addison-Wesley, 1949), Gabaix (QJE, 1999), Gabaix (AER, 1999), Gabaix and Ioannides (2004), Gabaix (AER, 2011), etc), firms (Okuyama et al (Physica A, 1999), Axtell (Science, 2001), Fujiwara et al (Physica A, 2004)), banks (Aref and Pushkin (2004)), and mutual funds (Gabaix et al (QJE, 2006)).   I say purported, because experience has taught me never to believe in a power-law that I haven't estimated myself!

In this post, I am going to provide evidence against the power-law as an appropriate model for mutual-funds using the data from the same source as Gabaix et al (QJE, 2006).  The figure below shows two survival plots of the size, as measured in terms of $-value of assets under management, of U.S. mutual funds at the end of 2009 using data from CRSP.1  The top panel shows the entire data set, the second panel shows only upper 20% of mutual funds (roughly those funds with assets under management greater than $1 billion) and is intended to match as closely as possible Figure VII from Gabaix et al (QJE, 2006).
Choosing a threshold to include only the largest 20% of mutual funds for a given year, Gabaix et al (QJE, 2006) report an average estimate for the power-law scaling exponent of ζ ≈ 1 (or α ≈ 2) over the period 1961-1999.   Gabaix et al (QJE, 2006) estimate α using OLS on the upper CDF of the mutual fund distribution (although they report similar results using the Hill estimator).  

Using my larger data set I estimate, via OLS and choosing the same 20% cut-off criterion (which leaves 1313 observations in the tail), a scaling exponent of ζ = 1.11 (or α ≈ 2.11). Here is a plot showing my OLS estimates:

I estimated the scaling exponent using maximum likelihood in two ways.  First, I apply the Hill estimator to the data using the same 20% cut-off as in Gabaix et al (QJE, 2006); second, I re-estimate the scaling exponent using the Hill estimator, while choosing the threshold parameter to minimize the KS distance as in Clauset et al (SIAM, 2009).  Method 1 obtains an estimate of α = 1.97(3); while method 2 obtains estimates of α = 2.04(3) and xmin = $1.12 billion (which leaves 1077 observations in the tail).  Note that the KS distance, D, for each maximum likelihood fits is smaller than the KS distance obtained using the OLS estimate of α. 
Numbers in parentheses show the amount of uncertainty in the final digit (obtained using a parametric bootstrap to estimate the standard error).

Parameter uncertainty is estimated using the bootstrap:
  • Using 20% cut-off and a parametric bootstrap, I estimate a se for α of 0.026 and a corresponding 95% confidence interval of (1.912, 2.013)   
  • Choosing xmin via  Clauset et al (SIAM, 2009) and using a parametric bootstrap, I estimate a se for α of 0.032 and a corresponding 95% confidence interval of (1.976, 2.098)   
  • Finally, choosing xmin via  Clauset et al (SIAM, 2009) and using a non-parametric bootstrap, I estimate a se for α of 0.059 and a corresponding 95% confidence interval of (1.932, 2.113);  se for xmin of $0.530 B and a corresponding 95% confidence interval of ($0.398 B, 1.332 B)
Note that in all three cases, the 95% confidence interval for the estimated scaling exponent includes α=2 (i.e., Zipf's "law").  So far so good for Gabaix et al (QJE, 2006).
However, what about goodness-of-fit? Good data analysis is a lot like good detective work, and it is important to collect as much evidence as possible, relevant to testing the hypothesis at hand, before passing judgement.  As stressed in Clauset et al (SIAM, 2009), an assessment of the goodness-of-fit of the power-law model is an important piece of relevant statistical evidence.  Here are my goodness-of-fit test results:
  • Using a 20% cut-off as suggested in Gabaix et al (QJE, 2006) along with the parametric version of the KS goodness-of-fit test I obtain a p-value of roughly 0.00 using 2500 repetitions, which suggests that the power-law model is not plausible.  
  • Choosing xmin via  Clauset et al (SIAM, 2009) and using the parametric version of the KS goodness-of-fit test I obtain a p-value of roughly 0.19 using 2500 repetitions, which suggests that the power-law model is plausible.  
  • Finally, choosing xmin via  Clauset et al (SIAM, 2009) and using the non-parametric bootstrap version of the KS goodness-of-fit test I obtain a p-value of roughly 0.02 using 2500 repetitions, which again suggests that the power-law model is not plausible.  
On the whole, I think these results are not very supportive of the power-law model.  Even though the power-law model remains plausible when  I choose xmin via Clauset et al (SIAM, 2009) and assess goodness-of-fit using the parametric version of the KS test, it is important to note that such an assessment is not properly taking into account the flexibility of the Clauset et al (SIAM, 2009) procedure in choosing the threshold parameter (along with estimating the scaling exponent).2  Once I take this the additional flexibility into account (i.e., by using the non-parametric KS test), I again find that the power-law model is not plausible!  Here is a nice set of density plots of the bootstrap KS distances from each version of the goodness-of-fit test, that illustrates the differences between the parametric and non-parametric procedures (I hope!):
Note that implementing the non-parametric version of the KS goodness-of-fit test basically shifts and "condenses" the sampling distribution of the KS distance (relative to both parametric versions).  Taking into account the additional flexibility of the Clauset et al (SIAM, 2009) procedure for fitting the power-law null model reduces both the mean and variance of sampling distribution of the KS distance, D. 

Quick test of alternative hypotheses.  A very plausible alternative distribution for mutual funds is the log-normal (recall Gibrat's law of proportionate growth would predict log-normal).  Can I reject the power-law in favour of the log-normal using likelihood ratio tests?  YES!
  • Using a 20% cut-off as suggested in Gabaix et al (QJE, 2006) the Vuong LR test statistic is -3.63 with a two-sided p-value of roughly 0.00 (which implies that, given the data, I can distinguish between the power-law and log-normal) and a one-sided p-value of roughly 0.00 (implying that I can reject the power-law in favour of the log-normal!  
  • Choosing xmin via  Clauset et al (SIAM, 2009) the Vuong LR test statistic is -2.27 with a two-sided p-value of roughly 0.023 (which implies that, given the data, I can distinguish between the power-law and log-normal) and a one-sided p-value of roughly 0.012 (implying that I can reject the power-law in favour of the log-normal!
What are the economic implications or all of this?  Does it matter whether or not mutual fund size is distributed according to a log-normal or power-law distribution?

I think it matters quite a bit for the model put forward in Gabaix et al (QJE, 2006)! In Gabaix et al (QJE, 2006) investors take as given that the distribution of investors' size follows a power-law  Specifically, an investor makes use of the distribution of investor size in calculating his optimal trading volume.   Gabaix et al (QJE, 2006) relies on the power-law being a "good approximation" to the true distribution of investor size in order to justify investors taking a power-law distribution as given.  I have provided evidence that the power-law is not a plausible model, and that a log-normal distribution is a significantly better fit.  If the true distribution is not a power-law, then agents in Gabaix et al (QJE, 2006) are effectively solving a mis-specified optimization program and there is no longer any guarantee that the solution to the properly specified optimization program will result in power-law tails for equity and volume (paradoxically, however, this might turn out to be "good" for Gabaix et al (QJE, 2006) in the sense that I have argued in previous posts that the tails of equity returns are not power-law anyway!).

However, whether or not it matters if a distribution is log-normal, power-law, or simply "heavy-tailed" depends on context.  In this case a log-normal distribution is consistent with Gibrat's law of proportionate growth.  Gibrat's law applied to investor size says that if the growth rate of investors' assets under management is independent of the amount of assets currently under management, then the distribution of investor size will follow a log-normal distribution.  One could easily test whether or not the growth rate of mutual funds is independent of size. Maybe someone already has?
    Personally, I think the important takeaway from the above analysis is just that there is quite extreme heterogeneity in the size of investors (although not extreme enough to justify a power-law)!  In other words, the distribution of investor sizes is generically "heavy-tailed."  Investors are not necessarily small relative to the "market" which suggests that at least some investors are unable to take prices parametrically (i.e., as given) when determining their optimal trading behavior.  In this respect I wholeheartedly agree with Gabaix et al (QJE, 2006): investor size does play a significant role in determining dynamics of asset prices.  These results also suggest an alternative way to think about the liquidity of an asset.  An asset might be very liquid (i.e., re-saleable) for one investor, but might be very illiquid for another because the desired volume of trade is different!  Liquidity might not simply be an inherent property of the asset itself, but may also depend on the "size" of the investor holding it!  

    1 Gabaix et al (QJE, 2006) use data on mutual fund assets from 4th quarter of 1999, whereas I use the larger and more recent data set from 4th quarter of 2009.
    2 Assessing goodness-of-fit using a parametric version of the KS goodness-of-fit test that takes the optimal threshold chosen using the Clauset et al (SIAM, 2009) method as given is both conceptually easier to understand, and computationally simpler to implement.  This procedure also sets an effective lower bar for the plausibility of the power-law model: if the power-law model is not plausible using this parametric KS goodness-of-fit test, then it will be even less plausible if I use the more flexible (and more rigorous) non-parametric KS goodness-of-fit test.  

    Sunday, February 26, 2012

    Can a better choice of threshold save the power-law?

    In my previous post, I provided evidence against the power-law as a plausible model for the tails of equity returns.  In my analysis I took as given that the "true" value of the threshold parameter for the power-law model was 2 in all three cases.  In this post I am going to do away with the assumption of a threshold of 2 and see if it changes the analysis in anyway.  After all it could be that  Gabaix et al (Nature, 2003) consulted a sub-standard Oracle who gave them bad information regarding the true power-law threshold! 

    I will now re-estimate α while choosing the threshold parameter to minimize the KS distance between the data and a true power-law as suggested in Clauset et al (SIAM, 2009).  As a baseline, the KS distance for the best-fit power-law model from my previous post where xmin =2 were D=0.01994, 0.0293, and 0.0199 for the combined, positive, and negative tails, repsectively.

    The following are my new parameter estimates for α and xmin along with a minimal KS distances, D, using the Clauset et al (SIAM, 2009) procedure to choose the threshold parameter:

    α = 4.51(2); xmin = 4.282; D = 0.00836
    α+ = 5.19(7); x+min = 6.267; D = 0.0141
    α- = 4.25(2); x-min = 4.112; D = 0.00758

    Again, the numbers in parenthesis indicate the amount of uncertainty in the final digit, and again I use a parametric bootstrap to estimate 95% confidence intervals for the above estimates of the scaling exponent this time taking the estimated value of the threshold parameter as given.1 

    95% CI for α is (4.477, 4.551);  α+: (5.067, 5.325); α-: (4.207, 4.298)

    These estimates for the scaling exponents are wildly different then those reported in Gabaix et al (Nature, 2003) and there is now a huge difference between the estimated scaling exponents for the positive and negative tails of the combined return distributions!  Looks like the Oracle gave Gabaix et al (Nature, 2003) a value for the threshold parameter that was much too low!  The result: significantly biased estimates of the scaling exponents in all three cases.  Here is a plot of the best-fit power-law model for the negative tail...
    The next, and perhaps most important, question to ask is whether or not this improved estimate of the scaling exponent alters the results of the goodness-of-fit tests. For the combined tails using 2500 replications, the p-value for the KS goodness-of-fit test is 0.0004; for the positive tail only the p-value is 0.1472; finally, for the negative tail the p-value is 0.0644.  Thus the power-law is rejected as plausible for the combined tails, remains plausible for the positive tail of the distribution, and is border-line rejected for the negative tail of the distribution.2

    Finally, when I test the power-law model against a log-normal alternative using likelihood ratio tests, I fail to reject the two-sided null hypothesis (i.e., that both the power-law and the log-normal are "equally far" from the truth) when I combine both the positive and negative tails into a single data set, and when I analyze the positive tail separately.  More plainly, given the data at hand I simply can not distinguish between the power-law model and the log-normal in either of these cases.

    However, for the negative tail of the return distribution, I am still able to distinguish between the power-law and the log-normal (Vuong statistic: -2.17, two-sided p-value: 0.03), and reject the power-law in favor of the log normal (one-sided p-value: 0.015).  

    At this point I feel like I need to reiterate that combining all of the returns from all equities listed on the Russell 1000 index and analyzing the distribution as if all of the observations came from a single company is of questionable value, and that the use of plots of the combined tails in Gabaix et al (Nature, 2003) and Gabaix et al (QJE, 2006) encourage readers of those papers to conclude that the support for the power-law model is much stronger than is justified (at least by my analysis).  In future posts I will assess support for the power-law model by analyzing each tail of each equity separately.  You might guess that the power-law does not come out well...and you would be right!

    1Note that these confidence intervals are likely too narrow as they ignore the uncertainty in my estimate of the threshold parameter (it would be ideal to use non-parametric bootstrap to derive standard errors and confidence intervals for the above estimates, but alas my 4 year old MacBook is too slow to handle so much data!). 
    2It is worth noting that although the power-law is a plausible model for the positive tail of the return distribution, there are comparatively few, only 4178 observations, above the optimal threshold. For comparison, the best-fit power-law for the negative tail had almost 19,000 observations above its optimal threshold.

    Thursday, February 23, 2012

    Power-laws, equity returns, and the fallacy of division...

    This recent post from the Three-Toed Sloth about led me to this Science article by Mason Porter et al provided much needed incentive to break from preparing to present my first-year research paper to write this post questioning the "stylized fact" of power-law tails in equity return distributions...
    The above plot below shows the empirical upper cumulative distribution of the absolute values of normalized daily returns taken from CRSP for the constituents of the Russell 1000 index (i.e., largest 1000 U.S. companies as measured by market capitalization) at the end of Q2 2011 (roughly 5.7 million observations).  I normalize the returns of each constituent stock so that the normalized returns have mean of 0 and a standard deviation of 1.  Specifically, I first construct a daily logarithmic return series, ri, for stock i using the closing price (adjusted for splits and dividends!).  I then define the normalized return on date t as follows:

    r'i,t = (ri,t - ri) / σi

    where ri,t is the logarithmic return on date t, ri is the mean and σi is the standard deviation of the daily logarithmic returns series for stock i.  This plot is my version, using CRSP data instead of TAQ data (only because my university does not subscribe to TAQ!), of Figure 1 in Gabaix et al (Nature, 2003) and Figure I in Gabaix et al (QJE, 2006).

    I have several issues with this plot.  First, Gabaix et al (Nature, 2003) and Gabaix et al (QJE, 2006) lay out a theory about the tail of the return distribution of individual stocks, not a population of stocks.  The above plot suggests that the tails of the return distribution of a population of stocks (in some average sense?), might be well approximated by a power-law.  It suggests nothing about how well the power-law model approximates reality for each on the constituent stocks individually.     
    Second, in plotting the empirical upper cumulative distribution of the absolute value of returns, one is assuming that the negative and positive tails are well approximated, by the same probability distribution.  But why should this be true?  My prior would be that the economic mechanism generating large negative returns, although perhaps related, would fundamentally different than the mechanism generating large positive returns.  Specifically, my prior would be that there would be, on average, negative skewness in the return distribution: the negative tail of stock returns would be "heavier" than the positive tail (on average at least).1  Gabaix et al (QJE, 2006) makes the claim that the power-law holds for both the positive and negative tails separately based on previous empirical research: Gopikrishnan et al (Physical Review E, 1999) and Plerou et al (Physical Review E, 1999) among others.  Gopikrishnan et al (Physical Review E, 1999)  analyse the return distributions of various global stock market indices and find evidence of power-law tails with estimated scaling exponents of ζ ≈ 3 (or α ≈ 4) for both tails.  Plerou et al (Physical Review E, 1999) analyse the positive and negative tails of the distribution of combined normalized returns for 1000 U.S. companies and report significantly different estimates of the scaling exponent for the positive and negative tails.  Plerou et al (Physical Review E, 1999) also report significant variation in estimated scaling exponents across stocks for both the positive and negative tails around an "average" value of ζ ≈ 3 (or α ≈ 4).  N.B. the estimates of the scaling exponents relied predominately on OLS methods, with occasional use of the Hill estimator popular in the quantitative finance literature.  No tests of goodness-of-fit of the power-law model.

    Finally, it seems a bit odd to simply combine all of the normalized returns for all stocks listed on the Russell 1000 index and analyse the distribution of these returns "as if" they represented returns of a single company.  Especially given that you can not treat returns from different companies as being independent as there are likely complex dependencies between returns of different stocks on any given day t.  Plerou et al (Physical Review E, 1999) seems to be a source of this practice (which they justify based on the visual similarities between empirical distribution functions and in order to get "better statistics").

    Just for kicks, I decided to apply the now familiar Clauset et al (SIAM, 2009) method for fitting power-law model to the above data.  First, I  consult the Oracle (which in this case was the same Oracle consulted by Gabaix et al (Nature, 2003)), and am told that 2 is a reasonable threshold for the power-law model.  Maximum likelihood estimation (AKA the Hill estimator) yields the following estimates of of the scaling exponent:

    α = 3.943(5); α+ = 3.976(7); α- = 3.908(7)

    where the numbers in parenthesis indicate the amount of uncertainty in the final digit.  The figure below shows the best-fit power-law using the Gabaix et al (Nature, 2003) threshold.

    I use a parametric bootstrap to estimate 95% confidence intervals for the above estimates (taking a value of the threshold parameter of 2 as given):

    95% CI for α is (3.932, 3.953);  α+: (3.963, 3.988); α-: (3.894, 3.922)

    All three confidence intervals exclude α=4. The estimated scaling exponent for the negative tail is less than the estimate for the positive tail, and the confidence intervals for α+ and α- are disjoint (a sufficient condition to insure that the estimates are significantly different), implying that the negative tail of the return distribution is significantly "heavier" than the positive tail.  Are these differences in parameter estimates economic meaningful?  I argue they are.

    Suppose that you are a portfolio manager.  You hold all 1000 stocks on the Russell 1000 in your portfolio, and are interested in estimating the probability that at least one of the stocks in your portfolio will experience a negative return of at least 80 standard deviations in magnitude.2  First suppose that you are well versed in the economics/econo-physics literature on power-laws and believe that the scaling exponent is definitely α = 4. Using an estimate for the scaling exponent of  α = 4, the probability of such an event is roughly 8.63e-7.  Now suppose you are well versed in the statistical literature on actually estimating power-laws from data (which means that you also know better than to abuse OLS!) and are able to estimate α = 3.943.  In this case the probability of such an event occurring is roughly 1.06e-6.  Finally, suppose that you know enough stats to estimate the scaling exponent correctly and enough economics to suspect that you should treat negative and positive returns differently.  You estimate a scaling exponent for the negative tail of α- = 3.908, and from this you estimate the probability of observing a negative 80 standard deviation event to be roughly 1.21e-6.  If probability of observing a negative 80 standard deviation return using α- = 3.908 is roughly 14% larger than the estimated probability obtained using α = 3.943 and is roughly 40% larger than the estimated probability obtained using α = 4.3  The point is that small changes in the estimate of the scaling parameter can lead to large changes in out-of-sample forecasting, and that these large differences in predicted probabilities are economically meaningful.

    So I have fit a power-law model to the data and used the results to argue against applying a "generic" power-law model to both tails of the return distribution.  Fine.  But, I have said nothing about goodness-of-fit.  Perhaps the power-law is a rubbish fit to the data, in which case the above probabilities are likely over-estimates of the true probability of a negative 100 standard deviation event. Using the same KS goodness-of-fit testing procedure advocated in Clauset et al (SIAM, 2009) method I find p-values of 0, 0, and 0 using 2500 replications in each case.  Looking into the bowels of the goodness-of-fit test, under the power-law null hypothesis the typical KS distance between the model and the synthetic data is over an order of magnitude smaller than the KS distance between the observed data and the best-fit power-law model (which was D=0.01994).  It would seem that the power-law is just not a good fit.

    If the power-law model is a poor fit to the data, then what better models might there be?  How about a log-normal?  Using likelihood ratio tests I am able to reject the power-law in favour of the log-normal ( Vuong statistic: -12.85; p-value: 8.09e-38!). 
    In case you are wondering, the probability of observing an 80 standard deviation event under the best-fit log-normal distribution is 6.89e-9.   Using the negative tail of the return distribution, an 80 standard deviation event is 175 times more likely under the best-fit power-law model than under the best-fit log-normal! 

    At some level, my biggest complaint about the figures in Gabaix et al (Nature, 2003) and Gabaix et al (QJE, 2006) is that they unintentionally encourage readers to engage in a kind of fallacy of division, and in doing so make it seem that the support for the power-law hypothesis is stronger than is clearly justified.  I plan to follow with additional posts related to my own work in this area (much of which forms the first chapter of my PhD thesis).  The next post will see how (if!) the results change when I apply methods from Clauset et al (SIAM, 2009) to select the optimal threshold (instead of consulting the Oracle).  Comments and constructive criticism are always welcome... 

    1One plausible economic mechanism for generating asymmetry in return distributions would be some sort of leverage effect.  For example, suppose that an investor is trading on margin using his current asset holdings as collateral.  If the value of his collateral drops, he may be faced with a margin call which might force him to sell some of his assets to reduce his leverage.  This "fire-sale" of assets will likely further depress asset prices.  Meanwhile, if the value of the underlying collateral rises then this investor is relatively un-constrained: he may choose to borrow more (against his now more valuable collateral) or he may simply choose to consume some of the proceeds, etc.  The basic idea is that if collateral values drop, and you are a highly levered investor, then your behaviour may be significantly constrained; while if you are highly levered, and the value of your underlying collateral rises, then you are relatively un-constrained.  I would suspect that this type of behavioural asymmetry would generate asymmetry in the return distributions of assets. 
    2Gabaix et al (QJE, 2006) reports observing returns of over 80 standard deviations in magnitude in the TAQ data set.  I just wanted some reasonable number that was larger than any return I observe in my data.
    3Although I think that these differences are suggestive, it would be better if I could translate these differences in probabilities into differences in something like bank capital requirements.  

    Wednesday, November 30, 2011

    Power-laws in Gold? A journey into the world of dependent data...

    Attention Conservation Notice: 
    1. I have a son!
    2. Though plausible, the power-law is probably not the best model for the positive tails of gold returns; power-law model is firmly rejected for the negative tail.  
    3. Rest of post is the long, tired ramblings about my travails in finding confidence intervals for my power-law parameter estimates with dependent data.
    Apologies for the long lag between posts, but my wife gave birth to our son Callan Harry on Sept. 19th and the joys of being a father took precedence over my blog posts!  Now that things have settled down a bit, I thought I would take the time to write up some thoughts on the price of gold... 
    The above top plot in the above chart displays the daily nominal price of gold from 2 January 1973 through 28 November 2011using historical data from usagold.com.   Note the sharp increase in the nominal price of gold following the creation of gold backed ETFs in March 2003.  The bottom plot displays normalized standard logarithmic returns for gold over the same time period.  Note the ever-present volatility clustering.

    Normally when looking at an asset price over such a long time period one would prefer to look at the real price of the asset rather than the nominal price.  As I am interested in exploring the effects of credit and liquidity constraints on asset prices, I want to focus on the dynamics of the nominal price of gold.  Credit constraints typically enter the picture in the form of debt contracts which I assume are written in nominal rather than real terms.  There are two ways to think about liquidity/re-saleability: quantity liquidity/re-saleability and value liquidity/re-saleability.  A measure of quantity liquidity might be the average daily trading volume of an asset over some time period.  A measure of value liquidity, might be the average $-value of an asset traded over some time period.  I feel like the two concepts are not identical, and I prefer to think of liquidity in terms of value re-saleability rather than quantity re-saleability.

    Show me the tails!
    In thinking about how the dynamics of asset prices might be impacted by credit and liquidity constraints, it seems logical (to me at least) that one should focus on the behavior of the extreme tails of the return distribution.  Below is a survival plot of both the positive and negative tails of normalized gold returns along with the best-fit power-law model found using the Clauset et al. (2009) method.
    This approach to parameter estimation assumes that the underlying data (at least above the threshold parameter) are independent.  How realistic is this assumption for gold returns (and for asset returns in general)?  The random-walk model for asset prices, which is consistent with the strong form of EMH, would suggest that asset returns should be independent (otherwise historical asset price data could be profitably used to predict future asset prices).  Given that gold returns, and asset returns generally, display clustered volatility one would expect that the absolute returns would display significant long-range correlations.   Below are plots of various autocorrelation functions for gold...
    Both the positive and negative tails of normalized gold returns display significant autocorrelations.  MLE of scaling exponents assumes that gold returns, above the threshold parameter, are independent.  My main concern is that the consistency of the MLE will be impacted by the dependency in the data. Perhaps the autocorrelations will go away if I restrict the sample to only observations above a given threshold?
    Dependency in the positive tail of gold returns does appear to go away as I raise the threshold (it almost disappears completely above the optimal threshold).  ACF plot for the negative tail is similar, although given that the optimal threshold is lower for the negative tail, the dependency is a bit more significant.  Perhaps this argues for using an alternative to criterion to the KS distance in selecting the optimal threshold parameter.  Perhaps Anderson-Darling?  Anderson-Darling tends to be conservative in that it chooses a higher threshold parameter which might help deal with data dependency issues.  But higher threshold means less data in the tail, which will make it more difficult to both reject a spurious power law and rule out alternative hypotheses.  Life sure is full of trade-offs!

    Given that the behavior of ACF is likely to vary considerably across assets, I would like to be able to make more general statements about the consistency of the MLE estimator in the presence of dependent data.  Any thoughts?

    Parameter Uncertainty...
    Following the Clauset et al. (2009) methodology, I derive standard errors and confidence intervals for my parameter estimates using a non-parametric bootstrap.  My first implementation of the non-parametric bootstrap simply generates synthetic data by re-sampling, with replacement, the positive and negative tails of gold returns, and then estimates the sampling distributions for the power-law model's parameters by fitting a power-law model to each synthetic data set.

    To give you an idea of what the re-sampled data look like, here is a survival plot of the negative returns for gold along with all 1000 bootstrap re-samples used to derive the confidence intervals...
    ...and here is a plot of the bootstrap re-samples over-laid with the best-fit power law models:
    The bootstrap replicates can be used to derive standard-errors for the scaling exponent and threshold parameters for the positive and negative tails, as well as various confidence intervals (percentile, basic, normal, student, etc) for the parameter estimates.  Which of these confidence intervals is most appropriate depends, in part, on the shape of the sampling distribution.  The simplest way to get a look at the shape of the sampling distributions is to plot kernel density estimates of the bootstrap replicates for the various parameters.  Dotted black lines indicate the MLE parameter estimate.

    WTF! Note that all of the sampling distributions have multiple local peaks!  I suspect that the multi-peaked densities are due to sensitivity of the optimal threshold to the resampling procedure.  Is this an example of non-parametric bootstrapping gone awry?  The fact the the bootstrap estimates of the sampling distributions have multiple peaks suggests that standard confidence intervals (i.e., percentile, basic, normal, student, etc) are unlikely to provide accurate inferences.

    The best alternative method (that I have come across so far) for calculating confidence intervals when the sampling distribution has multiple peaks is the Highest Density Regions (HDR) from Hyndman (1996).  Again, the dotted black lines indicate the MLE parameter estimates. 
    The (significant!) differences between the estimated densities in the simple kernel density plots and the HDR plots is due to the use of different estimation methods.  HDR uses a more sophisticated (more accurate?) density estimation algorithm based on a quantile algorithm from Hyndman (1996).  Bandwidth for density estimation is selected using the algorithm from Samworth and Wand (2010). The green, red, and blue box-plots denote the 50, 95, and 99% confidence region(s) respectively.  Note that sometimes the confidence region consists of disjoint intervals.  But I still don't like that the densities are so messy.  It makes me think that there might be something wrong in my resampling scheme.  Can I come up with a better way to do the non-parametric bootstrap sampling for asset returns?  I think so...

    My second implementation of the non-parametric bootstrap attempts to deal with the possible dependency in the gold returns data by using a maximum entropy bootstrap to simulate the logarithm of the gold price series directly.  I generate 1000 synthetic price series, and for each series I calculate the normalized returns, split the normalized returns into positive and negative tails, and then generate the sampling distributions for the power law parameters by fitting a power-law model to each tail separately.

    Here are some plots of synthetic gold price data generated using the meboot() package:
    I did not expect that the distribution of synthetic data would collapse as the number of bootstrap replicates increased.  Was I wrong to have this expectation? Probably! Is this a law-of-large numbers/CLT type result? Seems like variance at time t of the bootstrap replicates of the gold price is collapsing towards zero while the mean at time t is converging to the value of the observed gold price. Clearly I need to better understand how the maximum entropy bootstrap works! Update: this behavior is the result of my using the default setting of force.clt=TRUE within the meboot() function.  One can eliminate this behavior by setting force.clt=FALSE.  Unfortunately, I have not been able to find a whole lot of information about the costs/benefits of setting force.clt=TRUE or force.clt=FALSE!  The documentation for the meboot package is a bit sparse...

    However, a plot of the negative tails of the normalized returns calculated from the simulated price series does look like I expected (which is good because this is the synthetic data to which I actually fit the power-law model!)...
    The HDR confidence intervals based on the maximum entropy non-parametric bootstrap are much nicer!  Mostly, I think, because the densities of the bootstrap replicates are much more well-behaved (particularly for the scaling exponents!).
    Clearly I have more work to do here.  I really don't feel like I have a deep understanding of either the maximum entropy bootstrap or the HDR confidence intervals.  But I feel like both are an improvement over my initial implementation...

    Goodness-of-fit for the Power-Law?
    Using the KS goodness-of-fit test advocated in Clauset et al. (2009), I find that while the power-law model is plausible for the positive tail (p-value: 0.81 > 0.10), the power-law model can be rejected for the negative tail of normalized gold returns (p-value: 0.00 < 0.10).
    The KS goodness-of-fit test generates synthetic data similar to the empirical data below the estimated power-law threshold, but that follow a true power-law (with MLE for scaling exponent) above the estimated threshold.  This implementation destroys any underlying time dependencies in the data.  Are these underlying time dependencies important for understanding the heavy-tailed behavior of asset returns? Previous research suggests yes.  Suppose that time dependencies, in particular the slow-decay of volatility correlations, are key to understanding the heavy-tailed behavior of asset returns.  Does the current implementation of the goodness-of-fit test under-estimate the goodness-of-fit for the power-law model?

    Testing Alternative Hypotheses:
    I end this ludicrously long blog post with a quick discussion of the results of the likelihood ratio tests used to determine if some alternative distribution fits the data better than a power-law.
    Results are typical.  Even though the power-law model is plausible for the positive tail of gold returns, the power-law can be rejected at the 10% level in favor of the power-law with an exponential cut-off.  Also, other heavy-tailed alternatives, namely the log-normal and the stretched-exponential, can not be ruled out (even the thin-tailed exponential distribution can not be ruled out!).  For the negative tail, the power-law is simply not plausible, and the power-law with cut-off is preferred based on log-likelihood criterion.  All of these alternative distributions were fit to the data using maximum likelihood estimators derived under the assumption of independence of observations in the tail.  

    Code and data to reproduce the above results can be found here.

    Many thanks to Aaron Clauset for his recent post on power-laws and terrorism, which was a major inspiration/motivation for my writing this post.  Also, many thanks to Cosma Shalizi for continuing to provide excellent lecture notes on all things R.

    Thursday, July 28, 2011

    Some Confused Hypothesis Testing...

    Returning to a previous post on fitting a power-law to Metropolitan Statistical Area (MSA) population data.  First, I reported results the following inferences based on Vuong LR tests (see previous post for Vuong stats and p-values):
    • Two Sided Tests (Null hypothesis is that both distributions are equally far from the "truth"):
      • Fail to reject null hypothesis for power-law and log-normal.
      • Reject the null hypothesis for the Weibull.
      • Reject the null hypothesis for the exponential.
    • One-sided Tests (Null hypothesis is a power-law):
      • Fail to reject the null hypothesis of power-law compared with Weibull.
      • Fail to reject the null hypothesis of a power-law compared with an exponential.
    • Vuong LR test for nested-models (used only for comparing a power-law and a power-law w/ exponential cut-off):
      • Reject null hypothesis of a power-law in favor of a power-law with exponential cut-off.
    Additionally, if you compare the following log-likelihoods for the various models, you would also select the power-law with exponential cut-off as being the preferred model:
    • Power-law with exponential cut-off: -2143.809
    • Log-normal: -2144.513
    • Power-law: -2146.368
    • Weibull: -2173.089
    • Exponential: -2182.766
    Later (after starring at the above plot for quite awhile and convincing myself that the log-normal "looks" better than the power-law with cut-off) I came back to my work, and decided to try and write some code to conduct a non-nested Vuong LR test to compare a power-law with exponential cut-off and a log-normal distribution.  I then reported that I was able to rejected the null hypothesis for the two-sided test (that both the power-law with exponential cut-off and log-normal distributions are equally far from the "truth"); and reject the null-hypothesis of a power-law w/ exponential cut-off for the one-sided test in favor of the log-normal.

    I may have been a bit hasty in that conclusion.  Upon further review, I think that there is either a bug in my code, or that I have made some conceptual error in implementing the test.  For one, I certainly should have noticed that the p-values for both tests were suspiciously small (they were both 0.00!) given the difference in their respective LR of less than 1! 

    As always comments are very much encouraged...

    Thursday, July 21, 2011

    Zipf's Law for cities holds...w/ a cutoff! Check that...Nope! Log-Normal!

    Update!  I have posted the code for this post online.  My code repository is very much a work in progress...so I hope you don't mind the clutter! 
    Continuing my quest to master the fitting of power laws to empirical data, today I am going to look at some results from Gabaix (1999) on the distribution of "city" sizes in the U.S. using a data set on the population of Metropolitan Statistical Areas.

    When fitting any distribution to a data set it is important to consider whether or not the variable in question is discrete, or continuous.  This is especially true when fitting a Power Law as Clauset et al (2009) demonstrate that assuming a discrete data set can be "well approximated" by a continuous distribution power-law is questionable (at best), and that parameter estimates obtained from fitting a continuous power-law to a discrete data set can be wildly inaccurate.  However, a continuous power law will be a good approximation to the data, if the data satisfy certain (rather general) conditions:
    1. The minimum value of the data set is greater than 1000, and
    2. There are more than 100 observations in the data set
    The above MSA  data set satisfies both criteria, so I will proceed using the continuous ML estimator (even though technically population is a discrete variable).

    Population of Metropolitan Statistical Areas (MSAs):
    I begin with a survival plot for the population of U.S. Metropolitan Statistical Areas (MSAs) using the data set from the Bureau of Economic Analysis (by way of Cosma Shalizi).
    I present estimation results using pareto.fit() (which is available for download) to implement both the continous MLE and the Old School regression method (used by Gabaix (1999) and many, many, other papers for comparison purposes).  Note that 1) n is the number of observations in the data set above the threshold, xmin; 2) for both the continuous MLE and the regression CDF methods, I used the KS procedure to find the threshold parameter, xmin.

    α xmin n
    Continuous MLE 1.99 321,400 146
    Regression CDF 2.07 344,600 143

    95% Confidence intervals for the continuous MLE are calculated using the same non-parametric procedure described in my previous post (recall the importance of using the non-parametric bootstrap to replicate the fitting of both parameters simultaneously in order to get consistent estimates of standard errors and confidence intervals).  Results...
    • α: (1.74, 2.22)
    • xmin: (137,600, 552,030)
    Note, that in keeping with Zipf's law (a surprisingly poor explanation of which can be found on Wikipedia), the point estimate for α is almost exactly 2.

    Goodness-of-fit tests (again using the Clauset et al (2009) KS procedure) yield a p-value of 0.20 > 0.10 which means that I fail to reject the null hypothesis of a power-law for the population of MSA!
    However, before I dance a little jig and claim evidence of Zipf's law I need to see if some other distribution fits the data as well or better than the power law.

    Power Law Log-Normal Exponential Weibull Power Law w/ Exponential Cut-Off
    Vuong Statistic NA -1.27 2.47 2.93 LR: -2.56
    p-value (two-sided) 0.20 0.20 0.01 0.00 p-value: 0.02

    Table turned out to be a bit muddled: I report two-sided p-values for the test of alternative hypotheses of log-normal, exponential, and stretched exponential (Weibull).  Conclusion: Fail to reject null hypothesis that the power-law and the log-normal are equally "bad" (in the sense of being same "distance" from the truth); Reject Exponential in favor of the Power-Law; Reject Weibull in favor of the Power-Law (Note that Vuong statistics are positive!).

    The kicker: I reject the Power-Law model in favor of a Power-Law with exponential cut-off!  Note that since these two distributions are nested, the power-law with exponential cut-off will ALWAYS fit better than a simple power-law.  The question is whether or not the improved fit is significantly better...in this case it looks like it does!
    Not sure where this blog post fits into the literature in economics on Zipf's law.  For example, I don't have any intuition about the type of theoretical mechanism that might generate a power-law with exponential cut-off.  Also, suppose, as above, I find that the data reject a pure power-law in favor of a power-law with exponential cut-off.  How do I know that the rejection isn't due to the exponential cut-off capturing "finite size" effects that must show up in the data at some point?  

    To conclude this very long post, I would like to point out an interesting forthcoming paper in the American Economic Review (co-authored by Xavier Gabaix).  The authors use a clustering algorithm to construct economic "cities" (which they argue are different from "political" cities) from very high resolution population data for the U.S. and UK.  They fit a power law to this data and find strong evidence in favor of Zipf's Law...I will let you take a guess as to which method they use to fit the power law (HINT: The answer makes baby Gauss cry!).

    UPDATE: So I felt like I should also test the null hypothesis of a power-law with an exponential cut-off against a log-normal alternative.  Apparently, Gibrat's Law applied to cities predicts a log-normal distribution.  I wrote some very quick and dirty code to try and conduct the test based on the same Vuong LR procedure for non-nested models that I used above.  Hopefully I didn't make any mistakes!  The results: Vuong Statistic: -3.21; p-value (two-sided): 0.00; p-value (one-sided): 0.00!  

    It's pretty late, and been a long day, but I think that this implies that I reject the power-law with exponential cut-off in favor of the log-normal distribution. Which is good news for proponents of Gibrat's Law! Weird that testing procedure implies that log-normal is preferred to power-law with exponential cut-off; power-law with exponential cut-off is preferred to power-law; and yet some how test is indifferent between log-normal and power-law!
    Finally, I am not confident that I plotted the fitted log-normal properly.  I had to multiply the density function by (3 + n/N) where n is the number of observations above xmin, and N is the total number of observations to get the plot to look correct.  I have not idea where the 3 comes from!  The (n/N) bit I understand...and now I sleep... 

    Update to the Update: Plotting issue fixed!  Was stupidly using the wrong function to evaluate probability density function for plotting purposes!  Plot still looks the same as above.  Thankfully, using the correct version of the density function removes the need to include the mysterious 3!  Only need include the usual factor of (n/N).

    Wednesday, July 20, 2011

    Power Law in Wealth Distribution? NO...

    Recently I have been teaching myself to estimate power laws from empirical data using the improved techniques developed in Clauset et al (2009) with the hopes of applying them in my own research.  What follows is an example where I apply these techniques to some data on the wealth of the 400 Richest Americans from 2003.  This example is partially covered in one of the co-author's of Clauset et al (2009) (i.e., Cosma Shalizi) lecture notes.

    Below, I expand on the example given in those lecture notes by
    1. Estimating the power-law threshold parameter, xmin, from the data
    2. Constructing standard errors and confidence intervals for the scaling parameter, α, and for the threshold parameter, xmin (that take into account the fact that I have estimated both α and xmin).
    3. Testing an alternative hypothesis for the distribution of wealth (i.e, log-normal)
    Always plot your data! I think that it is a good idea to start any data analysis with a plot of some description.  I will start this one with a plot of the probability of an individual having wealth greater than wealth (W), against wealth (W)...or more succinctly: a Survival Plot.
    If the data follow a power law, then this relationship should be roughly linear.  The converse, however, is not true!  Linearity is not a sufficient condition for a power law.  Also, lots of things might look linear on a log-log plot even when they aren't.

    Recipe for Fitting Power Laws:

    Parameter Estimation: Following Clauset et al (2009), I estimated the scaling parameter, α, using the method of maximum likelihood whilst using the suggested KS minimization procedure for estimating the threshold parameter, xmin.  Results:
    • α: 2.34
    • xmin: 9e8 ($900 million)
    • obs.above.threshold: 302 (out of 400)
    The code for estimating power laws can be found on Aaron Clauset's website.  There are two different implementations of the estimation procedure in R: pareto.fit() and plfit().  I have experimented with both, and although they give identical results for the above estimation, this is not true in general.  From what I can tell, the (slight) differences in estimation results stem from differing implementations of the KS test.  The pareto.fit() method uses R's ks.test() method, while plfit() hand-codes the KS test.  I have not (yet) conducted a Monte-Carlo study to see if the differences between the implementations are worth worrying about.

    Standard Errors and Confidence Intervals: To calculate the standard errors and confidence intervals for the parameters I used a non-parametric procedure outlined in Cosma's lecture notes.  Cosma also graciously provides the code for his lecture notes as well!  To calculate the standard errors and 95% confidence intervals for α and xmin, I had to alter (trivially) the provided code to take into account the fact that I was not given the threshold parameter and instead estimated it from the data.  Results:
    • α.se: 0.10
    • xmin.se: 364e6 ($364 million)
    • α.ci: (2.13, 2.53)
    • xmin.ci: (-2e8, 1.05e9)
    Obvious one cannot really have negative wealth (at least not at the level of $-200 million), so just note that the 95% confidence interval for xmin includes zero.

    Survival Plot (fitted power-law included):
    Looks pretty good!  But while you might be able to rule out a particular model just by looking at it, I think you would be hard pressed to rule a model "in" by "eye-balling it" (unfortunately, I am sure that people do). 

    Goodness-of-fit Test: Continuing to follow Clauset et al (2009), I again used a Monte Carlo KS test procedure to conduct a goodness-of-fit test for the power law model.  The procedure basically generates synthetic data by re-sampling the data itself below xmin, and then generating simulated data from a power law with the estimated parameters above xmin.   A power-law is then re-estimated to each synthetic data set and the KS statistic is stored.  The result is a p-value that represents the fraction of simulations whose KS statistic was at least as large as the one obtained from fitting the power law model to the wealth data.  A small p-value (i.e., a p-value < 0.10) indicates that only a very small fraction of the simulations had a KS stat as large as the wealth data and is evidence against the plausibility of the power-law model.  Results: p-value of 0.0012!  It would seem that the data reject the power-law as a plausible model.  This is perhaps not surprising...

    Testing Alternative Hypotheses:  So the wealth distribution does not follow a power law!  What about log-normal? I test this using the likelihood ratio procedure recommended in Clauset et al (2009). 

    The null hypothesis for comparing two non-nested distributions is that both distributions are equally far (in a Kullback-Leibler relative entropy sense...note to self: understand what the hell Kullback-Leibler relative entropy means!) from one another.  I report the Vuong test statistic and the two-sided p-value:
    • Vuong Statistic: 0.21
    • p-value: 0.83
    Conclusion: fail to reject the null hypothesis that the power-law and the log-normal distributions are equally bad (again in the Kullback-Leibler relative entropy sense).

    Other alternatives (for which  Clauset et al (2009) publish results) are the exponential, stretched exponential (Weibull), and the power-law with exponential cut-off.  I continue to have trouble getting the code for fitting the exponential distribution to work (keep getting an error in the nlm() method for minimizing the negative log-likelihood function...it keeps returning non-finite values...which makes me think that it is probably user error on my part).  I have also not yet figured out how to get the code for estimating the power-law with exponential cutoff to work.  This is at the top of my to-do list as I feel it is a very plausible alternative distribution (perhaps not necessarily for wealth...but definitely for some other data sets that I am working with at the moment).

    I can however report results for the stretched exponential (Weibull).  I was able to get the code to work after I learned how to suppress warnings in R (i.e., using the suppressWarnings() method).  Again I report the Vuong test statistic and the two-sided p-value:
    • Vuong Statistic: 8.05
    • p-value: 0.00
    The fact that the sign on the Vuong Statistic is positive indicates that the log-likelihood for the power-law model was greater than the log-likelihood for the stretched exponential (Weibull).   The test statistic of 8.05 is so large that the p-value is actually 8.88e-16!  Thus we can reject the stretched exponential in favor of the power-law for this data set (but given that we have already rejected the power law...this test is somewhat superfluous).  I hope I have interpreted the above test correctly.  If anyone feels otherwise definitely speak up! 

    The real objective of this post was to replicate results from fitting a power-law to empirical data (in a rigorous way that does not abuse linear regression), so that I feel confident in applying these tools in my own research.  Thus I purposefully did not stray too far from Clauset et al (2009).  I think I succeeded (at least in replicating the work of Clauset et al (2009) for this data set).  Hopefully, I have not made any glaring errors...but if I did, they are my own!  

    Friday, October 22, 2010

    Python Related Activities...

    Today I am debugging my code for power-law estimations...hopefully I will have working beta version by tonight!  If/when I do, I will post it somewhere for others to use...

    Also, I downloaded Sage today...it will go on my list of things Python related things to learn.

    Tuesday, October 19, 2010

    Remiss on my Posting Again...

    Teaching is hard work (and also quite time consuming)!  A quick update on my research, with more to follow later today...

    For the last two weeks, I have been teaching myself how to program in Python.  I have now done a quick pass through tutorials for all of the major modules that I suspect I will be using:
    1. NumPy
    2. SciPy
    3. NetworkX
    4. PyGraphviz
    5. SymPy
    6. Matplotlib
    7. RPy2
     I now plan to double back and do more detailed work with each of the packages goals in mind.  The first thing I plan to do is implement the Clauset et al (2009) procedure to detect power laws in  empirical data using Python and then systematically apply the procedure to major economic variables.  The Clauset et al procedure relies on MLE and an iterative application of the Kolmogorov-Smirnov (KS) test to determine the threshold value above which the power law is a good model.  The authors are heavily critical of least-square approaches to fitting power laws to data and detail the issues with using such procedures.  Interestingly most empirical studies of power laws in economics (that I have seen) seem to use ad hoc least-squares approaches.

    Whether the results of this study will be publishable or not I have no idea, but I think it would be very useful to have an understanding of which economic variables exhibit power law behavior, and which are simply heavy-tailed (i.e., log-normal or something else)...if nothing else I will add a new tool to my arsenal that has significant practical applications...