Equity returns: where power-laws go to die? Maybe...

A major criticism of previous empirical work assessing support for the power-law as a model for equity returns is that very few (any?) of the studies assess goodness-of-fit of the power-law model.  This post summarizes the results of goodness-of-fit testing for the power-law as a model for equity returns and is a continuation of my previous posts on the power-law as a model for equity returns using data on U.S. equities listed on the Russell 1000 index.

I implement two versions of the KS goodness-of-fit tests suggested in Clauset et al (SIAM, 2009):

• Parametric version: conceptually and computationally simple; assumes estimated x_min is the "true" power-law threshold and simulates data under the null hypothesis of a power-law with exponent equal to estimated α
• Non-parametric version: conceptually less straight-forward and computationally very intensive because it makes use of the fact that  x_min is estimated from the data.

Results:
Using the parametric version of the KS goodness-of-fit test I am able to reject the power-law model as plausible (i.e., p-value ≤ 0.10) for 17% of positive tails and 12% of negative tails.  I would not describe these results as overwhelmingly "anti-power-law," but I would remind readers that the parametric version of the goodness-of-fit test sets a lower-bound on support for the power-law (see my discussion of goodness-of-fit results for mutual funds for more details).  Here is a histogram of the p-values for the positive and negative tails of equities in my sample:

The results obtained using the more flexible non-parametric KS test are much less supportive of the power-law model.  I reject the power-law model as plausible for roughly 44% of positive tails and 37% of negative tails.  Here is another histogram of the goodness-of-fit p-values:

Clearly the non-parametric goodness-of-fit results are more damning for the power-law model than the parametric results.  But why? 

Discrepancy could be due to sample size effects.  The average number of tail observations for a given stock in my sample is 313 (positive tail) and 333 (negative tail): using daily data, I simply do not have that many tail observations to work with.  Because of the small sample size I need to rely the extra flexibility of the non-parametric version of the goodness-of-fit to be able to reject the power-law model as plausible.  I suspect that this would not be the case if I had access to the TAQ data set used by Gabaix et al (Nature, 2003).  It is also worth noting that the more data I get, the more likely I am to reject the power-law as plausible.  Specifically, for those equities for which I reject the power-law model as plausible, either for positive or negative tail (or both), I have a larger average number of both total observations and observations in the tail.  This sample size dynamic has been something I have observed fairly consistently whilst fitting and testing power-law models to various data sets.  The more data I obtain, the less support I find for a power-law and the more support I tend to find for heavy-tailed alternatives (particularly the log-normal). 

While I interpret the above results of goodness-of-fit tests as being decidedly against the hypothesis that the power-law model is "universal," clearly the power-law is a plausible model for either the positive or negative (or both) tails of returns for some stocks.  In my mind this immediately suggests that there might be meaningful heterogeneity in the tail behavior of asset returns and that an interesting research direction might be to explore economic mechanisms that could generate such diversity of tail behavior. 

I end with a significant disclaimer.  I am concerned that by ignoring the underlying time dependence (think "clustered volatility" and mean-reversion) in large returns that my goodness-of-fit test results might be biased against the power-law model.  Suppose that Gabaix et al (Nature, 2003) are correct about equity returns having power-law tails.  Given the dependence structure of returns, it could be the case that the typical KS distance between the best-fit power-law model and the "true" power-law is larger than what I estimate in implementing an iid goodness-of-fit test. 

If this is the case, then perhaps the reason I reject the power-law model so often is because my observed KS distance is obtained from fitting a power-law model to dependent data, whereas my bootstrap KS distances are obtained by fitting a power-law to synthetic data that follows a "true" power-law but ignores the underlying dependence structure of returns.  In order to address this issue, I will need to develop an alternative goodness-of-fit testing procedure that can mimic the time dependence in the returns data!  Fortunately, I have some ideas (good ones I hope!) on how to proceed...      

Zipf's Law does not hold for mutual funds!

Gabaix et al (Nature, 2003) and Gabaix et al (QJE, 2006) lay out an economic theory of large fluctuations in share prices based, in part, on the assumption that the size (as measured in dollars of assets under management) of investors in asset markets is well approximated by Zipf's law (i..e., a power-law with scaling exponent ζ ≈ 1 or α ≈ 2).  Zipf's law has been purported to hold for cities ( Zipf (Addison-Wesley, 1949), Gabaix (QJE, 1999), Gabaix (AER, 1999), Gabaix and Ioannides (2004), Gabaix (AER, 2011), etc), firms (Okuyama et al (Physica A, 1999), Axtell (Science, 2001), Fujiwara et al (Physica A, 2004)), banks (Aref and Pushkin (2004)), and mutual funds (Gabaix et al (QJE, 2006)).   I say purported, because experience has taught me never to believe in a power-law that I haven't estimated myself!

In this post, I am going to provide evidence against the power-law as an appropriate model for mutual-funds using the data from the same source as Gabaix et al (QJE, 2006).  The figure below shows two survival plots of the size, as measured in terms of $-value of assets under management, of U.S. mutual funds at the end of 2009 using data from CRSP.¹  The top panel shows the entire data set, the second panel shows only upper 20% of mutual funds (roughly those funds with assets under management greater than $1 billion) and is intended to match as closely as possible Figure VII from Gabaix et al (QJE, 2006).

Choosing a threshold to include only the largest 20% of mutual funds for a given year, Gabaix et al (QJE, 2006) report an average estimate for the power-law scaling exponent of ζ ≈ 1 (or α ≈ 2) over the period 1961-1999.   Gabaix et al (QJE, 2006) estimate α using OLS on the upper CDF of the mutual fund distribution (although they report similar results using the Hill estimator).  

Using my larger data set I estimate, via OLS and choosing the same 20% cut-off criterion (which leaves 1313 observations in the tail), a scaling exponent of ζ = 1.11 (or α ≈ 2.11). Here is a plot showing my OLS estimates:

I estimated the scaling exponent using maximum likelihood in two ways.  First, I apply the Hill estimator to the data using the same 20% cut-off as in Gabaix et al (QJE, 2006); second, I re-estimate the scaling exponent using the Hill estimator, while choosing the threshold parameter to minimize the KS distance as in Clauset et al (SIAM, 2009).  Method 1 obtains an estimate of α = 1.97(3); while method 2 obtains estimates of α = 2.04(3) and x_min = $1.12 billion (which leaves 1077 observations in the tail).  Note that the KS distance, D, for each maximum likelihood fits is smaller than the KS distance obtained using the OLS estimate of α. 

Numbers in parentheses show the amount of uncertainty in the final digit (obtained using a parametric bootstrap to estimate the standard error).

Parameter uncertainty is estimated using the bootstrap:

• Using 20% cut-off and a parametric bootstrap, I estimate a se for α of 0.026 and a corresponding 95% confidence interval of (1.912, 2.013)   
• Choosing x_min via  Clauset et al (SIAM, 2009) and using a parametric bootstrap, I estimate a se for α of 0.032 and a corresponding 95% confidence interval of (1.976, 2.098)   
• Finally, choosing x_min via  Clauset et al (SIAM, 2009) and using a non-parametric bootstrap, I estimate a se for α of 0.059 and a corresponding 95% confidence interval of (1.932, 2.113);  se for x_min of $0.530 B and a corresponding 95% confidence interval of ($0.398 B, 1.332 B)

Note that in all three cases, the 95% confidence interval for the estimated scaling exponent includes α=2 (i.e., Zipf's "law").  So far so good for Gabaix et al (QJE, 2006).
However, what about goodness-of-fit? Good data analysis is a lot like good detective work, and it is important to collect as much evidence as possible, relevant to testing the hypothesis at hand, before passing judgement.  As stressed in Clauset et al (SIAM, 2009), an assessment of the goodness-of-fit of the power-law model is an important piece of relevant statistical evidence.  Here are my goodness-of-fit test results:

• Using a 20% cut-off as suggested in Gabaix et al (QJE, 2006) along with the parametric version of the KS goodness-of-fit test I obtain a p-value of roughly 0.00 using 2500 repetitions, which suggests that the power-law model is not plausible.  
• Choosing x_min via  Clauset et al (SIAM, 2009) and using the parametric version of the KS goodness-of-fit test I obtain a p-value of roughly 0.19 using 2500 repetitions, which suggests that the power-law model is plausible.  
• Finally, choosing x_min via  Clauset et al (SIAM, 2009) and using the non-parametric bootstrap version of the KS goodness-of-fit test I obtain a p-value of roughly 0.02 using 2500 repetitions, which again suggests that the power-law model is not plausible.  

On the whole, I think these results are not very supportive of the power-law model.  Even though the power-law model remains plausible when  I choose x_min via Clauset et al (SIAM, 2009) and assess goodness-of-fit using the parametric version of the KS test, it is important to note that such an assessment is not properly taking into account the flexibility of the Clauset et al (SIAM, 2009) procedure in choosing the threshold parameter (along with estimating the scaling exponent).²  Once I take this the additional flexibility into account (i.e., by using the non-parametric KS test), I again find that the power-law model is not plausible!  Here is a nice set of density plots of the bootstrap KS distances from each version of the goodness-of-fit test, that illustrates the differences between the parametric and non-parametric procedures (I hope!):

Note that implementing the non-parametric version of the KS goodness-of-fit test basically shifts and "condenses" the sampling distribution of the KS distance (relative to both parametric versions).  Taking into account the additional flexibility of the Clauset et al (SIAM, 2009) procedure for fitting the power-law null model reduces both the mean and variance of sampling distribution of the KS distance, D. 

Quick test of alternative hypotheses.  A very plausible alternative distribution for mutual funds is the log-normal (recall Gibrat's law of proportionate growth would predict log-normal).  Can I reject the power-law in favour of the log-normal using likelihood ratio tests?  YES!

• Using a 20% cut-off as suggested in Gabaix et al (QJE, 2006) the Vuong LR test statistic is -3.63 with a two-sided p-value of roughly 0.00 (which implies that, given the data, I can distinguish between the power-law and log-normal) and a one-sided p-value of roughly 0.00 (implying that I can reject the power-law in favour of the log-normal!  
• Choosing x_min via  Clauset et al (SIAM, 2009) the Vuong LR test statistic is -2.27 with a two-sided p-value of roughly 0.023 (which implies that, given the data, I can distinguish between the power-law and log-normal) and a one-sided p-value of roughly 0.012 (implying that I can reject the power-law in favour of the log-normal!

What are the economic implications or all of this?  Does it matter whether or not mutual fund size is distributed according to a log-normal or power-law distribution?

I think it matters quite a bit for the model put forward in Gabaix et al (QJE, 2006)! In Gabaix et al (QJE, 2006) investors take as given that the distribution of investors' size follows a power-law  Specifically, an investor makes use of the distribution of investor size in calculating his optimal trading volume.   Gabaix et al (QJE, 2006) relies on the power-law being a "good approximation" to the true distribution of investor size in order to justify investors taking a power-law distribution as given.  I have provided evidence that the power-law is not a plausible model, and that a log-normal distribution is a significantly better fit.  If the true distribution is not a power-law, then agents in Gabaix et al (QJE, 2006) are effectively solving a mis-specified optimization program and there is no longer any guarantee that the solution to the properly specified optimization program will result in power-law tails for equity and volume (paradoxically, however, this might turn out to be "good" for Gabaix et al (QJE, 2006) in the sense that I have argued in previous posts that the tails of equity returns are not power-law anyway!).

However, whether or not it matters if a distribution is log-normal, power-law, or simply "heavy-tailed" depends on context.  In this case a log-normal distribution is consistent with Gibrat's law of proportionate growth.  Gibrat's law applied to investor size says that if the growth rate of investors' assets under management is independent of the amount of assets currently under management, then the distribution of investor size will follow a log-normal distribution.  One could easily test whether or not the growth rate of mutual funds is independent of size. Maybe someone already has?

Personally, I think the important takeaway from the above analysis is just that there is quite extreme heterogeneity in the size of investors (although not extreme enough to justify a power-law)!  In other words, the distribution of investor sizes is generically "heavy-tailed."  Investors are not necessarily small relative to the "market" which suggests that at least some investors are unable to take prices parametrically (i.e., as given) when determining their optimal trading behavior.  In this respect I wholeheartedly agree with Gabaix et al (QJE, 2006): investor size does play a significant role in determining dynamics of asset prices.  These results also suggest an alternative way to think about the liquidity of an asset.  An asset might be very liquid (i.e., re-saleable) for one investor, but might be very illiquid for another because the desired volume of trade is different!  Liquidity might not simply be an inherent property of the asset itself, but may also depend on the "size" of the investor holding it!  

¹ Gabaix et al (QJE, 2006) use data on mutual fund assets from 4th quarter of 1999, whereas I use the larger and more recent data set from 4th quarter of 2009.
² Assessing goodness-of-fit using a parametric version of the KS goodness-of-fit test that takes the optimal threshold chosen using the Clauset et al (SIAM, 2009) method as given is both conceptually easier to understand, and computationally simpler to implement.  This procedure also sets an effective lower bar for the plausibility of the power-law model: if the power-law model is not plausible using this parametric KS goodness-of-fit test, then it will be even less plausible if I use the more flexible (and more rigorous) non-parametric KS goodness-of-fit test.  

Can a better choice of threshold save the power-law?

In my previous post, I provided evidence against the power-law as a plausible model for the tails of equity returns.  In my analysis I took as given that the "true" value of the threshold parameter for the power-law model was 2 in all three cases.  In this post I am going to do away with the assumption of a threshold of 2 and see if it changes the analysis in anyway.  After all it could be that  Gabaix et al (Nature, 2003) consulted a sub-standard Oracle who gave them bad information regarding the true power-law threshold! 

I will now re-estimate α while choosing the threshold parameter to minimize the KS distance between the data and a true power-law as suggested in Clauset et al (SIAM, 2009).  As a baseline, the KS distance for the best-fit power-law model from my previous post where x_min =2 were D=0.01994, 0.0293, and 0.0199 for the combined, positive, and negative tails, repsectively.

The following are my new parameter estimates for α and x_min along with a minimal KS distances, D, using the Clauset et al (SIAM, 2009) procedure to choose the threshold parameter:

α = 4.51(2); x_min = 4.282; D = 0.00836
α⁺ = 5.19(7); x⁺_min = 6.267; D = 0.0141
α^- = 4.25(2); x^-_min = 4.112; D = 0.00758

Again, the numbers in parenthesis indicate the amount of uncertainty in the final digit, and again I use a parametric bootstrap to estimate 95% confidence intervals for the above estimates of the scaling exponent this time taking the estimated value of the threshold parameter as given.¹ 

95% CI for α is (4.477, 4.551);  α⁺: (5.067, 5.325); α^-: (4.207, 4.298)

These estimates for the scaling exponents are wildly different then those reported in Gabaix et al (Nature, 2003) and there is now a huge difference between the estimated scaling exponents for the positive and negative tails of the combined return distributions!  Looks like the Oracle gave Gabaix et al (Nature, 2003) a value for the threshold parameter that was much too low!  The result: significantly biased estimates of the scaling exponents in all three cases.  Here is a plot of the best-fit power-law model for the negative tail...

The next, and perhaps most important, question to ask is whether or not this improved estimate of the scaling exponent alters the results of the goodness-of-fit tests. For the combined tails using 2500 replications, the p-value for the KS goodness-of-fit test is 0.0004; for the positive tail only the p-value is 0.1472; finally, for the negative tail the p-value is 0.0644.  Thus the power-law is rejected as plausible for the combined tails, remains plausible for the positive tail of the distribution, and is border-line rejected for the negative tail of the distribution.²

Finally, when I test the power-law model against a log-normal alternative using likelihood ratio tests, I fail to reject the two-sided null hypothesis (i.e., that both the power-law and the log-normal are "equally far" from the truth) when I combine both the positive and negative tails into a single data set, and when I analyze the positive tail separately.  More plainly, given the data at hand I simply can not distinguish between the power-law model and the log-normal in either of these cases.

However, for the negative tail of the return distribution, I am still able to distinguish between the power-law and the log-normal (Vuong statistic: -2.17, two-sided p-value: 0.03), and reject the power-law in favor of the log normal (one-sided p-value: 0.015).  

At this point I feel like I need to reiterate that combining all of the returns from all equities listed on the Russell 1000 index and analyzing the distribution as if all of the observations came from a single company is of questionable value, and that the use of plots of the combined tails in Gabaix et al (Nature, 2003) and Gabaix et al (QJE, 2006) encourage readers of those papers to conclude that the support for the power-law model is much stronger than is justified (at least by my analysis).  In future posts I will assess support for the power-law model by analyzing each tail of each equity separately.  You might guess that the power-law does not come out well...and you would be right!

¹Note that these confidence intervals are likely too narrow as they ignore the uncertainty in my estimate of the threshold parameter (it would be ideal to use non-parametric bootstrap to derive standard errors and confidence intervals for the above estimates, but alas my 4 year old MacBook is too slow to handle so much data!). 
²It is worth noting that although the power-law is a plausible model for the positive tail of the return distribution, there are comparatively few, only 4178 observations, above the optimal threshold. For comparison, the best-fit power-law for the negative tail had almost 19,000 observations above its optimal threshold.

Power-laws, equity returns, and the fallacy of division...

This recent post from the Three-Toed Sloth about led me to this Science article by Mason Porter et al provided much needed incentive to break from preparing to present my first-year research paper to write this post questioning the "stylized fact" of power-law tails in equity return distributions...

The above plot below shows the empirical upper cumulative distribution of the absolute values of normalized daily returns taken from CRSP for the constituents of the Russell 1000 index (i.e., largest 1000 U.S. companies as measured by market capitalization) at the end of Q2 2011 (roughly 5.7 million observations).  I normalize the returns of each constituent stock so that the normalized returns have mean of 0 and a standard deviation of 1.  Specifically, I first construct a daily logarithmic return series, r_i, for stock i using the closing price (adjusted for splits and dividends!).  I then define the normalized return on date t as follows:

r'_i,t = (r_i,t - r_i) / σ_i

where r_i,t is the logarithmic return on date t, r_i is the mean and σ_i is the standard deviation of the daily logarithmic returns series for stock i.  This plot is my version, using CRSP data instead of TAQ data (only because my university does not subscribe to TAQ!), of Figure 1 in Gabaix et al (Nature, 2003) and Figure I in Gabaix et al (QJE, 2006).

I have several issues with this plot.  First, Gabaix et al (Nature, 2003) and Gabaix et al (QJE, 2006) lay out a theory about the tail of the return distribution of individual stocks, not a population of stocks.  The above plot suggests that the tails of the return distribution of a population of stocks (in some average sense?), might be well approximated by a power-law.  It suggests nothing about how well the power-law model approximates reality for each on the constituent stocks individually.     

Second, in plotting the empirical upper cumulative distribution of the absolute value of returns, one is assuming that the negative and positive tails are well approximated, by the same probability distribution.  But why should this be true?  My prior would be that the economic mechanism generating large negative returns, although perhaps related, would fundamentally different than the mechanism generating large positive returns.  Specifically, my prior would be that there would be, on average, negative skewness in the return distribution: the negative tail of stock returns would be "heavier" than the positive tail (on average at least).¹  Gabaix et al (QJE, 2006) makes the claim that the power-law holds for both the positive and negative tails separately based on previous empirical research: Gopikrishnan et al (Physical Review E, 1999) and Plerou et al (Physical Review E, 1999) among others.  Gopikrishnan et al (Physical Review E, 1999)  analyse the return distributions of various global stock market indices and find evidence of power-law tails with estimated scaling exponents of ζ ≈ 3 (or α ≈ 4) for both tails.  Plerou et al (Physical Review E, 1999) analyse the positive and negative tails of the distribution of combined normalized returns for 1000 U.S. companies and report significantly different estimates of the scaling exponent for the positive and negative tails.  Plerou et al (Physical Review E, 1999) also report significant variation in estimated scaling exponents across stocks for both the positive and negative tails around an "average" value of ζ ≈ 3 (or α ≈ 4).  N.B. the estimates of the scaling exponents relied predominately on OLS methods, with occasional use of the Hill estimator popular in the quantitative finance literature.  No tests of goodness-of-fit of the power-law model.

Finally, it seems a bit odd to simply combine all of the normalized returns for all stocks listed on the Russell 1000 index and analyse the distribution of these returns "as if" they represented returns of a single company.  Especially given that you can not treat returns from different companies as being independent as there are likely complex dependencies between returns of different stocks on any given day t.  Plerou et al (Physical Review E, 1999) seems to be a source of this practice (which they justify based on the visual similarities between empirical distribution functions and in order to get "better statistics").

Just for kicks, I decided to apply the now familiar Clauset et al (SIAM, 2009) method for fitting power-law model to the above data.  First, I  consult the Oracle (which in this case was the same Oracle consulted by Gabaix et al (Nature, 2003)), and am told that 2 is a reasonable threshold for the power-law model.  Maximum likelihood estimation (AKA the Hill estimator) yields the following estimates of of the scaling exponent:

α = 3.943(5); α⁺ = 3.976(7); α^- = 3.908(7)

where the numbers in parenthesis indicate the amount of uncertainty in the final digit.  The figure below shows the best-fit power-law using the Gabaix et al (Nature, 2003) threshold.

I use a parametric bootstrap to estimate 95% confidence intervals for the above estimates (taking a value of the threshold parameter of 2 as given):

95% CI for α is (3.932, 3.953);  α⁺: (3.963, 3.988); α^-: (3.894, 3.922)

All three confidence intervals exclude α=4. The estimated scaling exponent for the negative tail is less than the estimate for the positive tail, and the confidence intervals for α⁺ and α^- are disjoint (a sufficient condition to insure that the estimates are significantly different), implying that the negative tail of the return distribution is significantly "heavier" than the positive tail.  Are these differences in parameter estimates economic meaningful?  I argue they are.

Suppose that you are a portfolio manager.  You hold all 1000 stocks on the Russell 1000 in your portfolio, and are interested in estimating the probability that at least one of the stocks in your portfolio will experience a negative return of at least 80 standard deviations in magnitude.²  First suppose that you are well versed in the economics/econo-physics literature on power-laws and believe that the scaling exponent is definitely α = 4. Using an estimate for the scaling exponent of  α = 4, the probability of such an event is roughly 8.63e^-7.  Now suppose you are well versed in the statistical literature on actually estimating power-laws from data (which means that you also know better than to abuse OLS!) and are able to estimate α = 3.943.  In this case the probability of such an event occurring is roughly 1.06e^-6.  Finally, suppose that you know enough stats to estimate the scaling exponent correctly and enough economics to suspect that you should treat negative and positive returns differently.  You estimate a scaling exponent for the negative tail of α^- = 3.908, and from this you estimate the probability of observing a negative 80 standard deviation event to be roughly 1.21e^-6.  If probability of observing a negative 80 standard deviation return using α^- = 3.908 is roughly 14% larger than the estimated probability obtained using α = 3.943 and is roughly 40% larger than the estimated probability obtained using α = 4.³  The point is that small changes in the estimate of the scaling parameter can lead to large changes in out-of-sample forecasting, and that these large differences in predicted probabilities are economically meaningful.

So I have fit a power-law model to the data and used the results to argue against applying a "generic" power-law model to both tails of the return distribution.  Fine.  But, I have said nothing about goodness-of-fit.  Perhaps the power-law is a rubbish fit to the data, in which case the above probabilities are likely over-estimates of the true probability of a negative 100 standard deviation event. Using the same KS goodness-of-fit testing procedure advocated in Clauset et al (SIAM, 2009) method I find p-values of 0, 0, and 0 using 2500 replications in each case.  Looking into the bowels of the goodness-of-fit test, under the power-law null hypothesis the typical KS distance between the model and the synthetic data is over an order of magnitude smaller than the KS distance between the observed data and the best-fit power-law model (which was D=0.01994).  It would seem that the power-law is just not a good fit.

If the power-law model is a poor fit to the data, then what better models might there be?  How about a log-normal?  Using likelihood ratio tests I am able to reject the power-law in favour of the log-normal ( Vuong statistic: -12.85; p-value: 8.09e^-38!). 

In case you are wondering, the probability of observing an 80 standard deviation event under the best-fit log-normal distribution is 6.89e^-9.   Using the negative tail of the return distribution, an 80 standard deviation event is 175 times more likely under the best-fit power-law model than under the best-fit log-normal! 

At some level, my biggest complaint about the figures in Gabaix et al (Nature, 2003) and Gabaix et al (QJE, 2006) is that they unintentionally encourage readers to engage in a kind of fallacy of division, and in doing so make it seem that the support for the power-law hypothesis is stronger than is clearly justified.  I plan to follow with additional posts related to my own work in this area (much of which forms the first chapter of my PhD thesis).  The next post will see how (if!) the results change when I apply methods from Clauset et al (SIAM, 2009) to select the optimal threshold (instead of consulting the Oracle).  Comments and constructive criticism are always welcome... 

¹One plausible economic mechanism for generating asymmetry in return distributions would be some sort of leverage effect.  For example, suppose that an investor is trading on margin using his current asset holdings as collateral.  If the value of his collateral drops, he may be faced with a margin call which might force him to sell some of his assets to reduce his leverage.  This "fire-sale" of assets will likely further depress asset prices.  Meanwhile, if the value of the underlying collateral rises then this investor is relatively un-constrained: he may choose to borrow more (against his now more valuable collateral) or he may simply choose to consume some of the proceeds, etc.  The basic idea is that if collateral values drop, and you are a highly levered investor, then your behaviour may be significantly constrained; while if you are highly levered, and the value of your underlying collateral rises, then you are relatively un-constrained.  I would suspect that this type of behavioural asymmetry would generate asymmetry in the return distributions of assets. 
²Gabaix et al (QJE, 2006) reports observing returns of over 80 standard deviations in magnitude in the TAQ data set.  I just wanted some reasonable number that was larger than any return I observe in my data.
³Although I think that these differences are suggestive, it would be better if I could translate these differences in probabilities into differences in something like bank capital requirements.  

Logistic Regression and Stock Returns...

Attention conservation notice: Stock returns are non-stationary! Logistic regression, smoothing splines, generalized additive models are all interesting techniques and fun to play with...but stock returns are still non-stationary! 

Suppose instead of trying to predict tomorrow's stock return based on today's return, I just try to predict whether or not tomorrow's return will be positive.  One way to do this would be using logistic regression. The response variable will be an indicator variable that takes a value of 1 if tomorrow's return is positive, and 0 otherwise.  The name of the game is to model the probability of tomorrow's return being positive, conditional on the value of today's return.

Here is the R summary output for a logistic model of the S&P 500 data from Yahoo!Finance:

Call:
glm(formula = (tomorrow > 0) ~ today, family = binomial, data = SP500.data.v2)

Deviance Residuals:
   Min      1Q  Median      3Q     Max 
-1.647  -1.223   1.080   1.129   2.007 

Coefficients:
            Estimate Std. Error z value Pr(>|z|)   
(Intercept)   0.1106     0.0161   6.866 6.60e-12 ***
today         8.6524     1.6788   5.154 2.55e-07 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 21458 on 15513 degrees of freedom Residual deviance: 21431 on 15512 degrees of freedom AIC: 21435

Number of Fisher Scoring iterations: 3

Both the intercept and the coefficient on today's return are highly significant.  How do we interpret the coefficients?  If today's S&P 500 return was 0.0, then the intercept represents the logistic model's prediction for the likelihood that tomorrow's S&P 500 return is positive.  However to get the predicted probability I need to transform the coefficient from the logistic scale back to the probability scale:

exp( 0.1106) / (1 + exp( 0.1106)) = 0.527611 or roughly 53%

The logistic model predicts there is a 53% chance of tomorrow's return being positive given that today's return was zero (slightly better than a coin flip!). Suppose that the S&P 500 was down 10% today (i.e., today's return is -0.10)? The predicted probability of tomorrow's return being positive is:

exp( 0.1106 + 8.6524(-0.10)) / (exp( 0.1106 + 8.6524(-0.10))) = 

0.3198024 or roughly 32%

I plot the predicted probabilities (and their confidence bands) against today's return using the logistic model for the daily returns of the S&P 500 from January 1950 through this past Friday.  The predicted probabilities increase monotonically as today's return goes from negative to positive (i.e., the lowest probability of a positive return tomorrow follows large negative returns today, and the highest probability of a positive return tomorrow follows a large positive return today.  The results of the logistic model are, I think, inconsistent with mean-reverting returns.  I would think mean reverting returns would require a higher likelihood of a positive return tomorrow if today's return where large and negative.  Note that the confidence bands are mildly asymmetric, and that they narrow considerably where the bulk of the observed returns lie.

Similar plots for the FTSE-100 from 1984-2011, and the Straits Times Index (STI) from 1987-2011. The probability of tomorrow's FTSE-100 being positive given today's return is basically a coin-flip much of the time!

The predicted probabilities from the logistic model using data from the Straits Times Index (STI) in Singapore are more similar to the predicted probabilities for the S&P 500.

I would like to have some sense of whether the above logistic models are well-specified.  How might I go about validating the above logistic regressions?  One way would be to estimate and compare the fits of some more flexible models (i.e., smoothing splines, generalized additive models, etc).  If the logistic regression is well specified, then I would expect that the more flexible models should not give significantly different predictions than the above logistic model.  Here is the plot of the predicted probabilities and the 95% confidence bands of the logistic model for the S&P 500, a smoothing spline (blue), and a generalized additive model (red).

WTF!  The smoothing spline and the GAM have very similar predicted probabilities, particularly when today's return falls within [-0.05, 0.05] (i.e., over the bulk of the observed returns).  However, visually, neither the smoothing spline nor the GAM appear to match the logistic regression well at all!  

The plot is actually a bit misleading, because your eye is immediately drawn to the huge differences in predicted probabilities for the extreme tails of today's return.  However, there are only a handful of observations in the extreme positive and negative tails of today's return (check the data rug) and thus the predicted probabilities for the smoothing spline and GAM models are unlikely to be very precise in these regions.  Better to focus on the differences in the predicted probabilities between the spline/GAM models and the logistic model in the region around today's return equals zero (where the bulk of the data is located).  In this neighborhood, the curve of predicted probabilities for all three models is increasing (consistent, I think, with some type of trend-following dynamic).  Note, however, that for the smoothing spline and GAM models, the slope of the predicted probability curve is much steeper (suggesting a more aggressive trend-following dynamic?) compared to the logistic model.

Here are the plots for the FTSE-100 and the Straits Times Index (STI)...

The whole point of fitting the smoothing spline and the GAM was to determine whether or not the logistic model is well specified.  As mentioned above, if the logistic model is well specified, then spline/GAM models should not be significantly better fits to the data.  We can measure goodness of fits by comparing the deviance of the logistic model with that of the GAM (I am going to ignore the smoothing spline because it doesn't technically respect the probability scale).  Thus the observed difference in deviance between the logistic model and the GAM for the S&P 500 is the deviance for the null model (i.e., Logistic model) less the deviance of the alternative (GAM model).

21430.65 - 21348.16 = 82.49624

The observed difference in deviance for the FTSE-100 and the STI are 16.53285 and 12.30237, respectively.  A smaller deviance indicates a better fit, thus, as expected, the more flexible GAM is a better fit for the S&P 500, the FTSE-100, and the STI.  Are these observed differences in deviance significant?  Let's go to the bootstrap!  Basic idea is to use the bootstrap to generate the sampling distribution for the difference in deviance between the null and alternative models, and then see how often these bootstrap replicates of the difference in deviance exceed the observed difference in deviance.  If the fraction of times that the replicated difference in deviance exceeds the observed difference is high, then it is likely that the observed improvement in fit using the GAM is the result of statistical fluctuations and is therefore not significant. 

Results? Running the above bootstrap using 1000 replications yields p-values of 0.00, 0.02, and 0.06 for the S&P 500, FTSE-100, and STI, respectively.  The p-values indicate that the improvement in fit using the GAM model on data from the S&P 500 and FTSE-100 is significant at the 5% level, and that the improvement in fit using the GAM model is significant for the STI data at the 10% level.

What does all of this mean? Well, as far as this little modeling exercise is concerning, the results of the bootstrap specification test suggest that we should use a GAM instead of a logistic model.  Here are the plots of the GAM predicted probabilities with confidence bands for the S&P 500, the FTSE-100 and the STI.  Note how wide the 95% confidence bands for the predicted probability of tomorrow's return being positive are when today's return is either really negative or really positive.  This is exactly as it should be!  There just aren't enough observed extreme returns (positive or negative) to support precise predictions.

Could this be used to construct a useful investment stratagem? I think doubtful.  Compare the GAM model's predictions for the S&P 500 above, which make use of historical data from 1950-2011, to the GAM model's predictions using data from 1993-2011 for SPY, the widely traded ETF which tracks the S&P 500 (I assume that implementing a stratagem would involve trading some ETF like SPY).  I have also included the logistic regression and its 95% confidence bands because the bootstrap specification test fails to reject the logistic model in favor of the GAM (p-value of 0.50).

The two are substantially different.  The interesting little window of trend-following behavior is now gone.  Perhaps it was a historical artifact from pre-computer trading days of yore?  The negative slope of the predicted probabilities is consistent with mean-reversion in returns.  

The underlying problem with trying to predict the behavior of stock returns, is that they are non-stationary.  It's not just that the parameters of the "data generating process" for stock returns are changing over time, the entire data generating process itself is evolving over time.  When the underlying process itself is changing, getting more historical data is unlikely to help (and in fact is likely to make predictions substantially worse!)... 

Update: Code is now available!

So you want to predict a stock market...but, which one?

Attention conservation notice: Stock returns are non-stationary time-series and as such all of the analysis below is wasted in a feeble attempt to squeeze water from the proverbial rock.

Which of the following equity markets would you expect to be more "predictable" using only information on past returns: the S&P 500, the FTSE-100, or the Straits Times index?  Personally, I had no really strong prior regarding which of these markets would be the most "predictable."  I suppose that a EMH purist (extremist?) would claim that they should all be equally unpredictable.

Some simple plots of S&P 500 logarithmic returns from 3 January 1950 - 24 August 2011 will help set the mood (all data is from Yahoo!Finance)...

Time series plot of the logarithmic returns exhibits clustered volatility (i.e., heteroskedasticity and serial correlation).  Kernel density plot is leptokurtic (i.e., excess kurtosis, which leads to heavier tails, than the best-fit Gaussian).  More on the scatter plot below...I will not bore you with the other plots (suffice it to say that they all exhibit the same properties).

A good place to start is a simple test of the random walk hypothesis.  If share prices follow a random walk with a drift, then tomorrow's share price should be the sum of today's share price and the drift term (and some statistical white noise).   Thus if we regress tomorrow's return on today's return, the coefficient on today's return should be statistically insignificant.

R_t+1 = μ + β R_t + ε _t+1

H₀: β = 0

Here is a scatter plot of FTSE-100 returns from 2 April 1984 - 24 August 2011.  The above regression specification is plotted in red, and the random walk null hypothesis is plotted in grey.  The two are virtually identical for the FTSE-100.

Although OLS parameter estimates are consistent in the presence of arbitrary heteroskedasticity and serial correlation,  OLS estimates of the standard errors are not.  To make sure my inferences are valid, I use HAC standard errors.  For the S&P 500 and the FTSE-100, using HAC standard errors, the coefficient on today's return is statistically insignificant.  However, even with HAC standard errors, the coefficient on today's return remains positive and significant for the Straits Times Index.  What this says to me is that predicting tomorrow's return using a linear model is pretty damned useless (unless you invest in Singaporean equities!).  But why should we limit ourselves to a linear model?  We shouldn't.

Above, I made the implicit assumption that the relationship between today's return and tomorrow's return is linear.  Is this assumption supported by the data?  One way we can test this assumption is to calculate the observed difference in the mean-squared error of the linear specification and that of a general non-linear specification.  One would expect that a general non-linear model would have a lower mean-squared error because it is more flexible, so we need to check whether this difference is statistically significant.  One way to do this is to generate a sampling distribution of this difference using the bootstrap, and then see how often we would observe a difference between the linear and non-linear specification as large as what we actually did observe.  If such a large difference is very rare, then this is evidence that the linear model is mis-specified.

Now, some care needs to be taken in generating the bootstrap replicates given that the time-series exhibits such complicated dependencies.  For my implementation,  I generate the simulated response variable by resampling the the residuals from the linear null model.

Linear or Non-Linear? That is the question...

For computational reasons, I use a smoothing spline to estimate my non-linear specification, using generalized cross-validation to choose the curvature penalty (you could easily use some other estimation method such as kernel regression in place of the smoothing spline).  Even though the smoothing spline has the lower MSE in all three cases, it is only significantly lower for the S&P 500.  The p-value for the bootstrap specification test (sketched above) is roughly 0.03, indicating that the linear specification can be rejected (p-values are 0.18 and 0.13 for the FTSE and the STI, respectively).

I plot the smoothing spline and 95% confidence bands (these are basic confidence intervals generated via a non-parametric bootstrap that re-samples the residuals of the smoothing spline) for both the S&P 500 and the Strait Times Index.  Since we are interested in prediction I plot the splines over a uniform grid of m=500 points (instead of simply evaluating and plotting the splines at the observed values of today's return).  Given that the smoothing spline is not significantly superior to the linear model for STI returns, I include the both the smoothing spline (blue) and the OLS regression (red) specifications in the scatter plot of STI returns.  Do these confidence bands seem too "tight" to anyone else?  Sample sizes are pretty large (over 15,000 obs. for the S&P 500)...

Could we use the above models to craft a profitable investment strategy?  Note that the OLS regressions for both the S&P 500 and the Straits Times Index have positive slope.  This is because, the bulk of the data (say for returns in [-0.02, 0.02]), exhibits mild "trend following" behavior.  Using the estimated smoothing splines, if today's S&P 500 return was -0.01 (or down 1%), 95% of the time tomorrow's return should be between -0.00156 (or down 0.156%) and -0.000657 (or down 0.0657%).  Similarly, if today's S&P 500 return was 0.01 (or up 1%), tomorrow's return should be between 0.000978 (or up 0.0978%) and 0.00188 (or up 0.188%).  Taking advantage of such minute expected returns would require, I think, either a large bankroll, or lots of leverage (or both!).

Also, what is the best way to trade the S&P 500 "market portfolio" in a cost efficient manner on a daily basis?  Exchange Traded Funds like the SPY might be an option.  Here is a plot of the smoothing spline fit to the data for SPY over the entire history of the fund (trading began on 29 January 1993).  I include the OLS regression fit because the slope is negative and significant (even using HAC standard errors) and because the spline fit is not a significant improvement over the linear model (p-value of 0.53 using my bootstrap specification test).

This plot of the SPY data from 1993 onwards looks quite a bit different, and the behavior of returns seems quite a bit different than the corresponding plot of S&P 500 data from 1950 onwards (despite the fact that SPY tracks S&P 500 almost identically from 1993 forward).  If the parameters of the underlying data generating process (DGP) driving stock returns remained constant over time, then one would expect to find that subsets of the data behaved similarly to the entire sample.  Maybe there are simply structural breaks in the data? Or, remember what I said about stock returns being non-stationary!

What about the FTSE-100?  Is all hope lost in trying to predict FTSE returns using past data? I may try predicting whether or not tomorrow's return will be positive given today's observed return using logistic regression in a future post (I am currently having difficulties getting R to give me the correct predicted values for the logistic models!  It worked fine yesterday!)

Update: Code is now available! 

Demand slopes UP...Supply slopes DOWN!

This post is a continuation of a previous post on liquidity and leverage (both posts closely follow the analysis of Adrian and Shin (2008)).

First, a quick summary of previous post.  There is a negative relationship between growth in balance sheets (value of assets) and leverage for passive investors such as households.  All data is taken from the U.S. Flow of Funds accounts maintained by the Federal Reserve.

However there is a positive relationship between growth in balance sheets (asset values) and leverage for security brokers and dealers (i.e., investment banks).  For this group, at least, leverage is pro-cyclical.

Continuing with the implications of pro-cyclical leverage.  Back to basic balance sheet arguments.  Consider a firm that actively manages it's balance sheet in order to maintain a constant leverage ratio of 10:1.  Start with the following balance sheet:

Assets	Liabilities
100	Equity: 10
	Debt: 90

Again assume that the value of debt is constant in response to change in asset values (which seems reasonable for small changes in assets values).  Now suppose that the value of assets increase by 1% to 101.  The new balance sheet is as follows (note that equity has increased by 10% as a result of the 1% rise in asset prices).

Assets	Liabilities
101	Equity: 11
	Debt: 90

The leverage ratio is now 101 / 11 = 9.18.  If the firm wants to maintain a leverage ratio of 10, it must take on more debt and use the cash it borrows to buy more assets.  How much debt?  Firm requires that 101 + D / 11 = 10, which implies that D=9.  Thus an increase in the value of assets of 1 leads to an increase in the quantity demanded of assets by the firm worth 9.  Price of asset goes up, quantity demanded of the asset by firm goes up.  The demand curve for assets slopes up!  The firm's new balance sheet now has a leverage ratio of 10.

Assets	Liabilities
110	Equity: 11
	Debt: 90

Same type of mechanism is at play for negative shocks to asset prices.  Doing the math leads to the conclusion that for a firm actively managing its balance sheet to maintain a constant leverage ratio, the supply curve for assets slopes down.  Note that just as equity increases as asset prices increase, equity bears the burden of adjustment when asset prices are falling.

This mechanical adjustment process of leverage will be strengthened if either:

1. Leverage is pro-cyclical
2. Asset markets are not completely liquid

If asset markets are not completely liquid, then the asset price will be affected by the change in the firm's demand for assets following an, asset price shock.  Consider a negative asset price shock.  The negative shock to asset prices leads, through the mechanism above, to the firm to sell assets in order to maintain the desired leverage ratio.  If markets are not perfectly liquid, then the firm's decision to sell assets will further decrease the price of these assets, which then weakens the firm's balance sheet even more, which causes the firm to sell more assets, and so the cycle goes.

Adrian and Shin (2008) outline empirical evidence consistent with the above amplification story.  Specifically they demonstrate that firm's balance sheet components are able to forecast changes in asset price volatility.  I am working to replicate their results (as they are highly relevant to my own empirical work on leverage and asset price volatility) and will follow up with the R code of the replication as soon written (hopefully in the near future!)...

More on Smoothing Splines...

Chalk yesterday's post up as a learning experience...following a very helpful comment left on yesterday's blog post, I made two changes to my code:

1. I am now calculate the confidence bands by bootstrapping the residuals (instead of the data points themselves).  The confidence bands yesterday were huge.  The reason why the confidence bands were so wide was because the bootstrap method I implemented scrambled the dataset during each iteration, which destroyed the time series nature of the underlying data.  Bootstrapping over the smoothing spline's residuals should preserve the times series nature of the data (which should narrow the confidence bands considerably). 
2. I am now using generalized cross validation (GCV) to pick the penalty for the curvature( for those following at home that is setting cv=FALSE using the function smooth.spline().

Here are the relevant plots using daily, weekly, and monthly returns:

Daily Returns:

Weekly Returns:

Monthly Returns:

Both the daily and monthly plots exhibit significant asymmetry between the left and right tails of the data.  Although I still wonder about the sensitivity of the results to certain outliers (particularly with the daily data).  Recall the EMH implies that returns should follow random walk with a drift, and the red-dotted line represents the EMH null prediction (ignoring the drift...feel free to mentally shift the dotted-red line up or down as you see fit). 

My (not very enlightening) interpretation of the above plots is that EMH works pretty well for the "body" of the data (all three plots are roughly "flat" where most of the data lies), but there is something fundamentally different governing the dynamics for large returns (and that whatever is governing the dynamics must effect large negative returns differently than large positive returns).  Note that the robust asymmetry is much harder (maybe impossible) to detect using a linear model. 

Today's Distraction...

For the past month or so I have been working my way through Cosma Shalizi's excellent course on data analysis.  Today I distracted myself from my own research by playing with Smoothing Splines (Cosma's Lecture 11).  Everything below was done in R.

First, I grabbed some historical data for the S&P 500 from Yahoo finance using the get.hist.quote() function from the tseries library.  I pulled down daily, weekly, and monthly data starting on 3 January 1950 and ending 31 Dec 2010 (start and end dates are for daily data).  I then constructed S&P 500 returns by taking the first-difference of the logarithm of the S&P 500 Adjusted Closing Price.   Here are (probably familiar) time series plots of the daily returns and a density plot...

Note that stock returns exhibit clustered volatility and are negatively skewed with significantly heavier tails than one would expect if returns were Gaussian.  On a side note (related to my current research), a couple of years ago some researchers at the Santa Fe Institute (specifically Stefan Thurner, J. Farmer, and John Geanakoplos) published a paper titled "Leverage Causes Fat Tails and Clustered Volatility."  Their model also predicts that returns should be negatively skewed (a point I think they should have included in the title).

Back to learning about splines! Is today's S&P 500 return useful in predicting tomorrow's S&P 500 return?  For the null hypothesis, I take a strongish form of the Efficient Market Hypothesis (EMH):

• Ho: Stock prices follow a random walk with a drift (i.e., returns should be mean zero white noise)

For alternative hypotheses, I use a brutally simple parametric model, and then what ever functional form the smoothing spline finds

• HA,1: r_t+1 = β₀ + β₁ r_t + ε_t
• HA,2: Whatever the smoothing spline kicks out

To test the null against the parametric alternative, we simply need to test the joint restriction that β₀=β₁=0.  Presumably to test the smoothing spline, we need to calculate some 95% confidence bands for the fitted spline and then look to see if the confidence bands contain the curve (really a line) predicted by the null hypothesis.  This is my first time using splines, so if anyone out there knows whether a better way (or a book) about how to do hypothesis testing with smoothing splines, I would be interested in hearing from you.

Here is a scatter plot of tomorrow's return against today's return.  I fit a simple linear regression to the data and plotted the curve in gray.  While both the slope and intercept terms are very significant (p-values essentially zero for both), it is worth noting that the standard confidence intervals are not valid (much too narrow) given the blatant violation of Gauss-Markov assumptions for the regression.  More work needs to be done before we can take this as evidence against the EMH null (since this post is about smoothing splines I am going to simply state that I would be surprised if, after calculating appropriate standard errors (either using bootstrapping, or some type of heteroskedastic robust standard errors, etc), the parameter results were still significant...but maybe!)

The smoothing spline is in orange.  I used the smooth.spline() function in R to fit the spline (using leave-one-out cross-validation to pick the optimal penalty for the curvature).  

If stock prices reflect all relevant information about the value of the stock, then one would expect that today's return should be pretty useless in predicting tomorrow's return (thus under the null the true regression line should be the dotted red line in the above scatter).

But what about the smoothing spline?  A few things:

1. While the regression line is positively sloped, the smoothing spline is negatively sloped for larger negative and large  positive values of today's return.
2. The asymmetry.  The slope of the smoothing spline is more negative for large negative values of today's return (compared with the slope of the smoothing spline for large positive values of today's return).
3. Outliers:  The October 1987 stock market crash looms large in the data. How sensitive is the estimated smoothing spline to these 1-2 observations?
4. Is the asymmetry of the smoothing spline a statistical artifact?
5. Most importantly, despite the dramatic appearance, is the smoothing spline significantly different than the dotted red line?

I am going to focus on point 5 for the rest of the post.  I calculated 95% confidence bands for the smoothing spline using a bootstrap re-sampling of the data points.  Basically, I re-sampled (with replacement) the stock return data creating a new synthetic data set, fit a smoothing spline to this new data set, and then repeated the process a bunch of times to build up a distribution that I could use to create the confidence bands.

For the most part, the dotted-red line lies entirely within the 95% confidence band for the smoothing spline (if you squint you can kind of see a small portion of the dotted-red line that lies outside the bands).  So despite the dramatic appearance of the smoothing spline I would say that we can not statistically distinguish it from the dotted-red line.

I was curious to see what the above plot might look like if I used weekly and monthly S&P 500 returns instead of daily returns...

Weekly Returns:

Monthly Returns:

I was surprised at how different the daily, weekly, and monthly smoothing splines turned out to be...still in all three cases the 95% confidence bands for the smoothing spline contain (almost completely) the red-dotted line.  I will have a think as to why they are so different, and perhaps follow up with another post.  My R code will be posted as soon as I have time to get my Google Code page up and running...until then feel free to email me (or leave email in a comment) and I will send it to you.

Update: As pointed out in a comment below, EMH predicts stock returns should follow a random walk with a drift...which implies that the dotted-red line doesn't necessarily need to have a zero intercept.  One would hope that the drift is slightly positive!