Can a better choice of threshold save the power-law?

In my previous post, I provided evidence against the power-law as a plausible model for the tails of equity returns.  In my analysis I took as given that the "true" value of the threshold parameter for the power-law model was 2 in all three cases.  In this post I am going to do away with the assumption of a threshold of 2 and see if it changes the analysis in anyway.  After all it could be that  Gabaix et al (Nature, 2003) consulted a sub-standard Oracle who gave them bad information regarding the true power-law threshold! 

I will now re-estimate α while choosing the threshold parameter to minimize the KS distance between the data and a true power-law as suggested in Clauset et al (SIAM, 2009).  As a baseline, the KS distance for the best-fit power-law model from my previous post where x_min =2 were D=0.01994, 0.0293, and 0.0199 for the combined, positive, and negative tails, repsectively.

The following are my new parameter estimates for α and x_min along with a minimal KS distances, D, using the Clauset et al (SIAM, 2009) procedure to choose the threshold parameter:

α = 4.51(2); x_min = 4.282; D = 0.00836
α⁺ = 5.19(7); x⁺_min = 6.267; D = 0.0141
α^- = 4.25(2); x^-_min = 4.112; D = 0.00758

Again, the numbers in parenthesis indicate the amount of uncertainty in the final digit, and again I use a parametric bootstrap to estimate 95% confidence intervals for the above estimates of the scaling exponent this time taking the estimated value of the threshold parameter as given.¹ 

95% CI for α is (4.477, 4.551);  α⁺: (5.067, 5.325); α^-: (4.207, 4.298)

These estimates for the scaling exponents are wildly different then those reported in Gabaix et al (Nature, 2003) and there is now a huge difference between the estimated scaling exponents for the positive and negative tails of the combined return distributions!  Looks like the Oracle gave Gabaix et al (Nature, 2003) a value for the threshold parameter that was much too low!  The result: significantly biased estimates of the scaling exponents in all three cases.  Here is a plot of the best-fit power-law model for the negative tail...

The next, and perhaps most important, question to ask is whether or not this improved estimate of the scaling exponent alters the results of the goodness-of-fit tests. For the combined tails using 2500 replications, the p-value for the KS goodness-of-fit test is 0.0004; for the positive tail only the p-value is 0.1472; finally, for the negative tail the p-value is 0.0644.  Thus the power-law is rejected as plausible for the combined tails, remains plausible for the positive tail of the distribution, and is border-line rejected for the negative tail of the distribution.²

Finally, when I test the power-law model against a log-normal alternative using likelihood ratio tests, I fail to reject the two-sided null hypothesis (i.e., that both the power-law and the log-normal are "equally far" from the truth) when I combine both the positive and negative tails into a single data set, and when I analyze the positive tail separately.  More plainly, given the data at hand I simply can not distinguish between the power-law model and the log-normal in either of these cases.

However, for the negative tail of the return distribution, I am still able to distinguish between the power-law and the log-normal (Vuong statistic: -2.17, two-sided p-value: 0.03), and reject the power-law in favor of the log normal (one-sided p-value: 0.015).  

At this point I feel like I need to reiterate that combining all of the returns from all equities listed on the Russell 1000 index and analyzing the distribution as if all of the observations came from a single company is of questionable value, and that the use of plots of the combined tails in Gabaix et al (Nature, 2003) and Gabaix et al (QJE, 2006) encourage readers of those papers to conclude that the support for the power-law model is much stronger than is justified (at least by my analysis).  In future posts I will assess support for the power-law model by analyzing each tail of each equity separately.  You might guess that the power-law does not come out well...and you would be right!

¹Note that these confidence intervals are likely too narrow as they ignore the uncertainty in my estimate of the threshold parameter (it would be ideal to use non-parametric bootstrap to derive standard errors and confidence intervals for the above estimates, but alas my 4 year old MacBook is too slow to handle so much data!). 
²It is worth noting that although the power-law is a plausible model for the positive tail of the return distribution, there are comparatively few, only 4178 observations, above the optimal threshold. For comparison, the best-fit power-law for the negative tail had almost 19,000 observations above its optimal threshold.