January 23, 2014

The issues with back testing a short stock strategy

I have been shorting stocks since 2006 using a quantified strategy that has remained relatively unchanged through the years. From 2006 to 2012, the strategy was one of my most consistent and profitable of all the strategies I have traded. I love shorting stocks because it is very hard psychologically, because of that, I believe that there is a good edge there. The test results have always bothered me because of the differences between back tested assumptions that sometimes are challenging to actually reproduce in real-world trading. Then in 2013 my fears became realized and all four fears below really hit me.

The issues with back testing shorting stocks

There are several issues with trading a short strategy that one cannot easily take into account when back testing. They are:

  1. Are shares available to borrow?
  2. Borrow interest
  3. Losses grow
  4. Overnight gaps – the Black Swan

Are shares available to borrow?

This is a constant problem. I use InteractiveBrokers and they are very good about having shares to borrow but there are still times when I cannot. Even with highly liquid stocks, one may not be able to borrow shares. In 2008, I could not borrow GM. Missed out on a good trade there. One could add a random possibility of shares not being available in the back test, but this does not really capture how real trading is. I will sometimes go months borrowing shares and then have several stocks in one week that I cannot borrow. Overall, missing these stocks has been a drag on my performance. In 2013, this was a much more common problem.

Borrow interest

I have had some insane borrow interest rates for some stocks. Sometimes it is next to nothing. Since there is no consistency on this, it makes it hard to model in back tests. One could use a constant value of borrow interest that would help make back tested results more realistic. In 2013, I saw so huge numbers.

Losses grow

One problem that has always bothered me with short trading is that as the position goes against you, it becomes a larger size of your portfolio which magnifies your losses. This is the opposite of a long trade. For example, if one shorts $10,000 of XYZ. It goes up 20% the first day, this means you lost $2000. The next day it goes up 25%, which means you lost an additional $3000, for a total loss of $5000. Now take a long position that does the same. The first day it loses 20% and then the next day it loses 25%. After the first day you are down $2000 and after the second day you are down a total of $4000. In 2013, this concept became real and painful.

Overnight gaps – the dreaded Black Swan

The large overnight gap is the biggest fear of any short trader.  If in one’s back tested results, a stock had gapped up 100%+, of course, one would not trade that system.  That’s because the overall system performance numbers would not look good and therefore, one would continue testing other parameters.

Because of this, I have realized that shorting stocks is similar to selling options. You may make money consistently for a long time, but all it takes is one trade to wipe out your profits from several months to years.

Here is the trade that did it to me in 2003.

140123A

This was looking like a perfect setup for my short system. Then two days later this happens.

140123B

Suddenly, months of profits are gone.  I had another trade almost as bad as this one happen later in the year.  Black swans showed up in 2013.

Final Thoughts

Given all the issues with short stock trading, I am really close to pulling the plug on my short strategy. Do you short stocks? How do you deal with these issues? What are your thoughts?

Backtesting platform used: AmiBroker. Data provider:Norgate Data (referral link)

Click Here to Leave a Comment Below

jpn - January 23, 2014 Reply

Bronte capital never shorts one stock for more than 1% i believe for this very same reason. and 2% of max capital in case of very high conviction. Or 2 en 4%. Anyway then even a 100% loss one one short will only be a 2 or 4% performance drag.

Only problem is, you need many, many short candidates.

    Cesar Alvarez - January 23, 2014 Reply

    I was using 3% position sizing. My overall portfolio did OK for 2013 but the short system was a definite drag on the performance. What I am saying that even if you are trading 1% capital, your results for the short strategy only may not make money (or not enough to make it worth trading) overall because of the 4 reasons in my article.

Seb - January 23, 2014 Reply

Depending on your country’s regulations, CFDs may solve the problems relative to available shares and repo rates. However, they may not be offered on a huge panel of stocks.

As for losses asymmetry, scaling out and back in a position at pre-determined intervals as the stock rises and drops back, respectively, is mathematically feasable but probably hard to test and involving serious brokerage costs.

Regarding overnight gaps, I do not see these as a material difference versus holding a stock long. In fact, they probably are less acute than a sharp drop since we know that, unless a corporate action sends a stock flyng, bear moves have a higher velocity than bull ones. Maybe blanketting a short position with far OTM call options may work but then again, it might be hard to backtest and would involve added costs in terms of theta decay, option spreads and plain broker fees.

Ronen - January 23, 2014 Reply

I pulled the plug on my equities short strategy in 2010. I didn’t have any trades that gaped up, but my experience was more along the line of the stock just continuously creeping up day after day for many days. Finally, when my exit was hit, the stock had advanced to such a degree that the reversion down was very minuscule. My loss was also compounded by margin interest. When I finally removed the strategy from the portfolio, it eliminated all my shorting profits and also dampened gains from other models.

Then, I decided to only short ETFs. The loses were smaller, but I still had the issue of losing trades just dragging on while margin interest piled up. Also, some ETF’s had some really crazy interest rates that turned a winning trade into a losing trade. Plus, you don’t know beforehand, if the trade you are about to enter has those outrageous rates unless you call the broker before hand.

Because of this, I’m being more selective with my shorts.

Your “loses grow” explanation is very interesting. I’ve never really thought of that side effect.

mhp - January 24, 2014 Reply

I’ve been a quantitative swing trader for 16 years. About half of my net revenue has come from the short side, which made me reluctant to abandon it even after many months of “slow bleed” kind of returns. As a result, I had my first ever net-negative year in 2013. Having said that, I have noticed that since about 12/1/13, the entire market has felt more “normal” (in a pre-QE sense), more short trades are working, and the short side net has been at least slightly positive.

As an aside, before this year, all of my trades were with-trend (buy breakouts and short breakdowns). After reading books by you and Connors, Howard Bandy and Keith Fitschen, I’ve not added a pair (long/short) of counter-trend systems. So far, I am very pleased with the increased smoothness of returns and lower per-trade risk that results from trading twice the number of systems at half the prior size.

As for outliers, how about ICPT? I had no position when it gapped up 200 points, but imagine having been short! After seeing that, and confirming with backtests it would be no great sacrifice to do so, I’ve decided to permanently exclude the drug company sector from my short side universe.

    Cesar Alvarez - January 24, 2014 Reply

    Yes, I saw ICPT the day it happened and was glad I was not short it. That was another nail in the short coffin for me. Yes, one could argue that would not happen if one dropped biotech but I am sure it has.

    In you comment did you mean to say that “I’ve added a pair…” It looks like you put in a ‘not.’

      mhp - January 24, 2014 Reply

      Yes, not sure where that ‘not’ came from! Meanwhile, I’m glad at the moment to have some short positions on…

Harold - January 24, 2014 Reply

Shorting is much more like buying options (volatility) than selling them (it). If you can break even (or even incur a small loss) in your shorts portion during bull markets then you should still keep it. Look at it as insurance protection against the inevitable fat tail events (1987, 2000, 2008, etc) that lurk in the future. Focus on how shorts help your performance during those periods. If you can break even or make money on the shorts during bull markets you should feel very lucky.

    Cesar Alvarez - January 24, 2014 Reply

    My short strategy looks for strongly uptrending stocks. So 2008 was a good year but bot great year for my short strategy. I feel there is too much difference between my test results and my trading results.

      Harold - January 25, 2014 Reply

      If real results diverge so much from backtest then it’s best to dump it — I agree.

Andy - January 25, 2014 Reply

One possible solution for these problems is to buy long using inverse ETFs. This will eliminate these problems. The only drawback I can see is that these ETFs are relatively new and not much history. I ran an initial quick backtest using a mean reversion strategy and the results are promising. With 2013 being such a bullish year, having this inverse ETFs strategy in 2014 can help reduce the swing in your portfolio in 2014. Here is a list of inverse ETFs I found

http://www.tradermike.net/inverse-short-etfs-bearish-etf-funds/

Andy

    Cesar Alvarez - January 25, 2014 Reply

    I will likely be looking into developing a short strategy using ETFs. I know that my current strategy does not work on ETFs.

James Villa - January 28, 2014 Reply

I totally agree with your assessments on the pitfalls of short strategies. My solution for this is what Andy alluded to, trading inverse ETFs. To overcome its short history I train my system on the prices of the underlying stock or index. Here’s how I do it , I train my system to generate trading signals based on the SP500 Index price history from 1995 to 2005 and forward test it from 2006 to present. I execute trades on Proshares 2x leverage, long on SSO for long signals and long on SDS for short signals. Since these ETFs started in 2006, the validation of the system is all out of sample. I am trading this live on my IRA account. This implementation also overcomes the limitations of IRA investing: (1) No Shorts and (2) No margins. Trading 2x leverage is equal to trading on 50% margin on stocks. Hoe this helps.

Harish - February 17, 2014 Reply

Hi Cesar,
Thank you for sharing your ideas/thoughts on the blog and other valuable information.
The lesson I have learned from shorting stocks for the past 10+ years is that the short position should be held for a very short time i.e. the trade should be closed on the same day.
Specially for the past 4+ years most of the day the market opens in green and closes in green. A study of the dow & or nasdaq and S&P EOD and the stocks that have trended down would probably give some kind of correlation/non correlation that can be used as a trading signal.
Most of the people believe in buying stocks, and on any dip the first reaction is “buy buy buy” though analytically it may be the wrong thing to do. The problem is sentiment wins analysis may times and there are very few who are analytically savvy and FE literate. In a long run analytical approaches is the way to go.
Harish

jose - April 27, 2014 Reply

With the feds running the printing presses at full steam, one would have to have nerves of steel to short stocks and fight the tide of monetary inflation flooding the markets. As an example, imagine trying to short the Caracas (or Zimbabwe) inflation-fed runaway markets.

Luke - July 8, 2014 Reply

I have looked at shorting ETF’s as an approach when the conditions become very overbought. Holding period is usually around 5-7 days. I am very much a newby but love the idea of having a shorting methodology long with a solid long swing trading strategy for smoothing returns. The reversion to the mean seems to work much better the more diversified the basket is and so that is why I have been more focuses on Index and sector etf’s rather than individual stocks.

    Cesar Alvarez - July 8, 2014 Reply

    Luke,

    I have just never found a ETF shorting strategy that gives good returns for the risk. I do miss shorting but I am sure most short mean reversion strategies (ETF or stock) are having a tough time this year.

Michael Grech - July 11, 2014 Reply

What percentage of capital would you think is appropriate to risk if one is shorting intraday only, using a stop loss? Besides high slippage would you worry about a stock gaping up intraday or possibly trading being halted and then opening much higher later?

    Cesar Alvarez - July 11, 2014 Reply

    That is an interesting question. I would look over the stock data over the past 5 years and see what the biggest intraday up moves were. The I would add 50% more to that. That would be the worst expected loss. From that I would determine my position sizing.

Marco - September 25, 2014 Reply

You are right. It’s really less risky only trading with the bulls.
http://nightlypatterns.wordpress.com

Jeff. F - April 28, 2015 Reply

This post points out some interesting issues associated with short selling. All to common unfortunately. The fact that short selling and borrowing shares involves more regulation than trading to the long side is something to consider.

Kirk - April 5, 2017 Reply

I stopped shorting stocks for the same reasons you mention, and went to selling call credit spreads, so my loss and gain are exactly defined. Since I don’t know where to get backtesting data for options, I subscribe to cmlvizdotcom’s CML trade machine option backtester to run options backtesting and choose the best combination of put credits spreads or call credits spreads.

    Cesar Alvarez - April 5, 2017 Reply

    Thanks for the info. I will have to check them out.

Aaron - June 11, 2018 Reply

How about futures? solves the easy to borrow question. Also a post on mean reversion specifically toward selling short would be very interesting especially since the parameters for a short strategy are so different from a long one, it would be interesting to hear your thoughts on that.

    Cesar Alvarez - June 11, 2018 Reply

    That really does not solve the problem for me because I short smaller cap stocks mostly.

James Dowsett-Cooper - June 15, 2018 Reply

So I am building an algorithm right now that shorts parabolics and you have definitely hit on some of my biggest fears.

1. Black Swan-
I think the best way to mitigate this issue is to just wire out money frequently and ensure you are actually taking profits. As long as you are doing that you should be able to make enough money in between the black swan events. Also ensuring you are shorting only on the backside will decrease the amount of gap ups but defintitely can’t be avoided fully.

2. Backtesting with borrow history
I am currently inquiring with IB to see if they can provide orderbook history as they have the same abilities as quantopian to do backtesting, and i think it is possible.

3. Interest
I am just going to research what are the top 20 highest interest rates interactive charges and use that as an estimate to be on the safe side

4. Losses growing
I am not too worried about this as I am going to have very tight stop losses as interactive commissions are dirt cheap (only issue is gap ups which will be mitigated by wiring out frequently)

5. Leverage variation (my own concern)
The amount you have to have in your account to trade certain stocks varies greatly, I am not sure how to account for this because for example what if you have a loss on a stock you can borrow a lot of but have a gain on a stock you could barely borrow any of, could be up on purely % gain but down to the amount of shares purchased. I am hoping IB’s backtesting will also take the history of leverage ability at that time into account.

Don’t give up on shorting, although it scary as long as you are wiring out there are lots of high % set ups out there!

    Cesar Alvarez - June 15, 2018 Reply

    Very good comments. I am trading shorting strategy right now.
    1. That does not help when stock gaps up 100%. It has happened to me. I keep the amount in the portfolio smaller.
    2. I would want it for delisted stocks. My data provider does not have this.
    3. I don’t think this is that big of an issue at least for the stocks I have been trading
    4. I have a stop too now but again does you no good on overnight gaps
    5. Not worried here because it is a small part of my overall portfolio.

Leave a Reply: