January 24, 2018

Equity Curve Monte Carlo Analysis

Imagine the following. You spent time developing a strategy with a compounded annual return of 24% and max drawdown of 18%. Profitable 10 of the last 11 years. An average 21 day rolling correlation with the SPY of .20.  Passes your out-of-sample testing. Passes your parameter sensitivity testing. Raise your hand if you would trade this? I would be the guy jumping up and down saying “yes!”.

Now you trade the strategy and the first year you lose money. Do you stop trading it or keep going? What about after two years, your average return is only 11%, half the backtested results? Do you stop trading it or keep going?

I would have a hard time trading it after two years. I can say I have done this before with strategies. Here is the issue, the above one year and two year scenarios are quite possible.

The Strategy

This strategy is based off a mean reversion strategy I am trading. I modified it so it could trade during all market conditions and for a CAR of around 20%. One thing I did not like about this variation was the low drawdown, which I will come back to later.

Equity Monte Carlo Traditional Method (Method 1)

One popular way of evaluating a strategy is through using Monte Carlo on the daily equity data. AmiBroker provides this functionality for free whenever you do a backtest. What this method does is each day randomly pick one of the daily returns from the entire backtest, with replacement. It does this for the number of days in your backtest and builds up an equity curve. Do this 1000 times and then you can look at the percentiles for the CAR and MDD. Here are the charts for this strategy.

Most people look at the 5% level to see what a “worst” case scenario is. For CAR at the 5% level you have 10.9%. This means that 95% of the time one can expect the 11-year CAR to be above this value.

The 50% percentile value of 17.5% is what I would consider a more realistic expectation of future returns. Our original return of 24% probably had a lot of luck involved.

Looking at the 5% level for Max Drawdown, we get a value of 24.8%. Meaning that 95% of our runs had a MDD of less than 24.8%. The median MDD is 16%.

There are several reasons I rarely use equity curve Monte Carlo analysis. The first is that we rarely trade for the entire backtest period before deciding if the strategy CAR is in line with expectations. On the MDD analysis, it tends to paint a more positive picture than other analysis.

Equity Monte Carlo without Replacement (Method 2)

Because of the under performance of a strategy I currently trade, I thought about other ways of how to analyze the equity curves.

Instead of replacing the daily returns after randomly picking one, we keep it out. The consequence of this is that the CAR of every Monte Carlos run is the same. But the MDD will be different because of the path to the final will be different. I like this because I have taken one part of the randomness out of the equation, the final CAR. This lets me focus on yearly returns and MDD. I used AmiBroker to create all the random equity paths and then used Excel to analyze the CAR, MDD and yearly returns.

Using this method, we get a much more negative few of MDD. The 5% level is now 36%. I mentioned at the beginning of the post that I did not like the fact that strategy had a low MDD of 18%. When I see a value that low, the path taken was probably lucky and that drawdown is low compared to what is likely. The median MDD is 25%. Both values are much worse than method 1.

Imagine trading this strategy and now you have a 25% drawdown. Besides not being happy, now do you keep trading or not? Using method 1, the 25% drawdown is at the 5% level. Meaning you would likely conclude this is what out of the ordinary and stop trading. Using method 2, this is only a 50% level move, so in this case you would probably not stop trading because of this analysis.

 

Yearly Returns

Here is a chart of the 11,000 yearly returns, 11 years per backtest times 1000 backtests.

The worst return is a bone jarring -35.5%. The 5% percentile level -8%. The 50 percentile is 23.6%.

Looking at the percentile returns for yearly returns of the strategy, we see they range from the 5th percentile to the 88th percentile.

Now to the question I asked at the beginning of the post. What do you do if after the first year of trading the strategy loses money? Using the Traditional Monte Carlo Method, you get no information about this from the traditional charts. One could generate the equity curves and so similar analysis.

Using Method 2, from the thousand runs, 12% had a first year loss. Having a losing year to start with is nothing to worry about.

What about having a two year return less +12%, this is half the strategy CAR. That happened 20% of the time. Quite possible to happen

Most people, myself included, would likely stop trading a strategy if the above happened. But as you can see these are not unlikely situations.

What about having a two year return under zero. That happened 4% of the time. If this happed, I would be concerned.

Large Market Years

I started this analysis because the SPY was up 21% last year and I was surprised that one strategy that I trade was not up more. When you trade a low correlation strategy, it’s hard to beat the market when it is up big. Looking at this strategy we can see there is a 45% chance of not beating the market. And a 26% chance you do not even do half the market return. These are both likely events and yet something that would cause people to stop trading the strategy.

Final Thoughts

This analysis assumes a strategy with low correlation to market. If the strategy highly correlates with the market, now one must consider how the market did along with how the strategy did for the year to get a better feel of how the strategy is doing compared to the market.

I rarely do this analysis because daily returns tend to be serially correlated and this type analysis ignores that. It is still useful to do because we can see how likely losing years are and underperformance. I prefer to use Sensitivity Analysis for my strategy evaluation.

Backtesting platform used: AmiBroker. Data provider: Norgate Data

Good quant trading,

 

Click Here to Leave a Comment Below

Akshay - January 24, 2018 Reply

Hi Cesar

Very informative post on MC analysis. I wanted to understand some of the key setting for MC on strategy which opens multiple positions at one time:

1) What kind of position sizing would you use for MC analysis, because generally we are using % equity for backtesting in amibroker and that might not be correct for MC analysis?
2) Do you use every day’s equity changes or trade list for MC analysis?
3)Your method-2 is just randomization of order of trades without replacement? So different paths to reach same equity level in the end.

Regards
Akshay

    Cesar Alvarez - January 24, 2018 Reply

    I am using the daily return of the portfolio to randomize. So in your questions 1 & 3 do not apply since I am using equity cahnges not trade list. I really do not like randomization of trade lists.

Dennis - January 25, 2018 Reply

Can you please explain what do exactly mean by the phrase: “What this method does is each day randomly pick one of the daily returns from the entire backtest, with replacement.”

What exactly do you mean with the second part “with replacement”?

Thanks in advance for explaining…

    Cesar Alvarez - January 25, 2018 Reply

    Here is an example that Ih hope helps. This is the week’s returns, +1.2%, +.5%, -.87%, +.33%, -1.9%.
    We have two ways of doing the Monte Carlo on this
    1- With replacement. Since I have 5 data points, I randomly choose a number between 1 and 5 five times. Say my five random numbers are 3,2,3,1,4, then my equity curve for the week would be: -.87%, +.5%, -.87%, +1.2%, +.33%. This means that every data point will likely not be used.

    2- Without replacement, I take the 5 days of returns and randomize the order of them. For example, +.5%, -1.9%, -.87%, +.33%, +1.2%. This means every data point will be used.

    Does this make sense?

dennis - January 25, 2018 Reply

Yes thanks a lot, now I understand it.

I also use Amibroker, but the second way can not be done in Amibroker right? Only the first way.

    Cesar Alvarez - January 25, 2018 Reply

    The second way, all the equity curves and yearly returns are generated in AmiBroker. The final analysis is done in Excel.

Akshay - January 25, 2018 Reply

Hi
Thanks for the answer. How would i do second method in Amibroker , could you give brief steps to follow to do in Ami ?

    Cesar Alvarez - January 25, 2018 Reply

    Not that easy to do. Here are the general steps. There is probably an easier way to do this but this was what first came to mind.
    1 – In my strategy code I use AddToComposite to export out the equity curve
    2 – In new file, I use Foreign to read in equity data
    3 – I randomize the order of the data
    4 – In low level CBT, I increase the daily cash amount by the daily return
    5 – CBT write out the yearly returns
    6 – Run this 1000 times
    Copy data to Excel to analyze

Akshay - January 26, 2018 Reply

Thanks for the reply.I will try it out. But i did not understand whats the difference between randomizing trade list and randomizing equity curve reruns..is it because you have multiple trades on same day so you don’t want to randomize list?

Akshay - January 26, 2018 Reply

Apologies for the typing error. “returns..is it because you have multiple trades on same day so you don’t want to randomize trade list?”

    Cesar Alvarez - January 26, 2018 Reply

    It has to do with position sizing mostly. A trade at the beginning might only be $10K but at the end now you are taking $20K sized. Doing trades works better when you are only trading one instrument which is not the case here

Jesús - January 27, 2018 Reply

Hello César, thanks for your articles.

I think the probability of two consecutive years below zero is 0.12 * 0.12 = 1.44%, right?

    Cesar Alvarez - January 27, 2018 Reply

    That is correct. Which is different from having a loss after 2 years.

Affordable Muscle Cars - April 21, 2018 Reply

Thank you for a wonderful overview and detailing of Monte Carlo Analysis.

Pieter-Jan - July 3, 2018 Reply

Hi Mr. Alvarez

Thank you for the great post! Could you give some more insight/your experience about how severe the sererial correlation is? I know Dr Bandy uses this in all of his works. Basically if this is not reasonable assumptions most of his work about trading system management falls apart I guess…

Thank you for your time!
Pieter-Jan

Kirk Dolan - February 3, 2022 Reply

Thanks, Cesar. That is an impressive analysis! I cannot improve on it. What I do when my Amibroker mean-reversion strategy cools off too much is go back to when it started underperforming, and then reoptimize a few of the settings, such as the % limit entry, the profit targets, exit at end-of-day versus open-next-day or take an average, and my exit criterion setting. I will also run the strategy in two separate “market conditions” to see if the settings change much. The market conditions could be connected to some technical levels of SPX or VIX, and could be called “bull market” and “bear market” to keep it simple. What I just described does not affect the inner workings of the code on how to get a list of qualified stocks for the next day, but only the % or ATR levels of entry and exit. For example, If I notice that the strategy did great through June, and has lagged from July to Nov., I will reoptimized some of these settings since July 1. I can recheck every couple of months. If the results are good, I’ll keep trading with the new settings. If not, I will stop trading it. This is not nearly as statistical as your excellent post, and I’m sure my method can be criticized, but it seems to help.

    Cesar Alvarez - February 4, 2022 Reply

    I am familiar with your method. I prefer not to be reoptimizing with that frequency. I may do this to a strategy that I stopped trading a while back ago in order to see if new parameters will work. Even that, I do not do that often. Glad it has been working for you.

Leave a Reply: