- in Mean Reversion , Research by Cesar Alvarez
Broken Strategy or Market Change: Investigating Underperformance
I recently had someone email me about the performance of a strategy I created back in late 2005/early 2006 and traded for a few years. I remember the strategy being a daily mean reversion set up with an intraday pullback entry. I figured it probably had not done well over the last decade. I stopped trading in the middle of 2008 because I did not like how it was behaving. In the backtest it did well in bear markets but was not doing so in the middle of 2008.
I ran the strategy from 2007, using the rules as they were published and was pleasantly surprised by the results. A CAR of 25%. Overall not too bad. Wish I had still been trading it. This is an eleven year out of sample test.
Then I remembered, didn’t this have insane results from 2000 to 2005. Here are those results.
Did you notice that 595% return in 2003? Now I am thinking what the heck happened since 2007? Look at the drop in yearly returns. Was the original strategy overfit and flawed? Did the markets change? If so, what is the difference? What follows is my investigation into trying to determine if the strategy is broken or the markets have changed.
Rules
I cannot share the rules because of the NDA I signed when it was created. What I can say is that it is a very typical mean reversion strategy that trades stocks.
I will include 2006 in the In-sample period because the return for the year was in line with previous years
Curve Fit?
Was the initial strategy curve fit? I do not have the original code so I made a best guess of the parameters I would have optimized on and their values. The strategy CAR was 1.3 standard deviations from the average of all the variations. Even the worst variation still had a very good CAR from 2000 to 2005 and saw similar drop in performance. This variation may have been a little overfit but that is not the entire story.
Number of Trades
Is there a decline in the number of trades? Average number of trades per year.
- 2000 to 2006: 171
- 2007 to 2016: 88
Wow, a drop of 50%! That starts to explain a lot. But why less trades?
Trading Universe
Are fewer stocks passing our liquidity filter? This the average per year.
- 2000 to 2006: 978,239
- 2007 to 2016: 861,763
A small decline but it does not explain the large drop.
Number of Setups
Since the strategy depends on intraday sell off, maybe we are seeing less setups. This is the average per year.
- 2000 to 2006: 5307
- 2007 to 2016: 2537
That explains the lower number of trades per year. We have less setups. This means we are seeing fewer stocks sell off and setting up for a mean reversion trade. This explains a lot. But let us see if it is the entire story.
Avg % P/L per Trade
Is the quality of the trade dropping too? Meaning what is the average % profit/loss per trade?
- 2000 to 2006: 6.25%
- 2007 to 2016: 2.73%
Anther drop of 50%. A double whammy. These last two stats explain the drop in the performance. But I wanted to know more.
Volatility
Are the trades less volatile and is that why they are not as big? These are the average 100-day historical volatility of the trades.
- 2000 to 2006: 116%
- 2007 to 2016: 106%
That is not leading to the decrease. What is causing the decrease in average % p/l? I don’t know what else to look at.
Final Thoughts
Is this strategy broken? I don’t think so. But what has changed is the market, which has greatly reduced the returns. Having fewer setups and smaller gains on the trades seems to have caused the reduction in returns. As to why, my theory is that it’s because of the popularity of mean reversion strategies and quantified trading. Have any ideas on what is going on or other tests you want to see? Post them in the comments below.
Remember we had 11 years of out-of-sample data to work with to determine if the strategy was originally overfit or if it had broken or if the markets had changed. Even with all this data it was not completely easy to figure out what was going on. The next time your strategy starts to perform poorly after 3 to 12 months, remember how hard it can be. I dropped the strategy after 6 months. It was not broken, the markets had changed.
Backtesting platform used: AmiBroker. Data provider: Norgate Data (referral link)