Why is the backtest loss higher than my Stop Loss? Is it a bug on your platform?

Since we have started AlgoTest.in, we’ve received countless messages from users asking how its possible that their backtest loss higher than stop loss value? Specifically, the questions take this form:

1. “How is it possible to have a loss greater than the SL that I put in my strategy?”
2. “If the leg-wise SL I put is 20 points, how is it possible to have a loss of 100 points??”
3. “My overall MTM SL was Rs. 5500, how can your platform show loss of Rs. 9000? Is there a bug in your platform?”

The answer is the same for backtests and the real world – yes it is possible! And no, it’s not a bug!

We would even go a step further and say, not only is it possible, but it is inevitable, in the fictional world of backtests, and in the world of actual trading. If any backtesting software or system tells you otherwise, either they’re misinformed, or they’re lying.

Backtests

In the world of backtests, we represent a constant flow of market data that your broker API receives as candles. A candle has the wonderful property of compressing all market activity over any period into 5 distinct data points – open, high, low, close and volume. With the benefit of this compression, however, comes a cost. The cost is loss of data. Once you go from 1-second granular LTP data to a 1-min candle, you’ve essentially distilled 60 price data points into 4.

This data loss can seem magnified during particularly volatile period, as it can introduce large discontinuities between the price of two candles. A cursory look at Zerodha chart shows an intra-day gap down discontinuity we found:

On 22 Feb’22 14:41, the close price of the candle was Rs 198.20, the open price of the next candle was Rs 196. If your Stop Loss was set between these prices, our platform assumes your SL got hit at the open price of the next candle. Here, the gap in the two candles isn’t much, only ~2 rupees. BUT, it can be much larger during periods of volatility.

Backtest loss never higher than stop loss? Beware!

We actually came across a user who showed us a backtesting report from some other service that showed an overall loss exactly equal to their MTM SL EVERY TIME the SL was hit. BUT, when we totalled the PnL manually at the individual leg level, we noticed that the actual loss was 2.5 times the reported MTM SL loss for some of the trades!

So if you ever see a backtest report where your loss reported on a trade is always less than or equal to the SL level you set, BE WARY! If you see such a backtest report,

Real world

In the real world as well, this is an inevitability. To keep it simple, here’s a tweet that shows an extreme scenario of what can happen in the market.

Our Recommendation

Now you can decide whether you want to use LTP or Candle Close for your Live Trades and Forward Tests. We recommend using the Candle Close setting in order to match more closely to your backtests. Again, as we have shared above, there are no guaranties that your backtest results and your live trade results will match. In fact it’s almost inevitable that they won’t. But this will help you get closer.

Click here for additional information.

Pros of Candle Close	Cons of Candle Close
– Your backtests and live trades will be executed on the same settings. This increases the chances of them being closer.	– Your backtests will be executed on Candle Close and your Live Trades will be executed on LTP. This will lead to more differences in the outcomes.

#freaktrade #37100PE a story in charts. y-axis represents the ltp, x-axis the time. These are all the trades in that one second. Someone shouted "FIRE" in a crowded theater with a narrow exit. Instead of an orderly evacuation, we had a stampede. pic.twitter.com/VJ65ghaGgo

— Raghav (@rogue_hft) September 3, 2021

— Raghav (@rogue_hft) September 3, 2021