ForumSections

Testing a daily loss limit EA: why "1 minute OHLC" gave me 105% and real ticks gave 90%

New comment
I was testing a daily loss lock in the Strategy Tester (100,000 account, 5% daily limit = 5,000, lock set at 90% of the limit).

With "1 minute OHLC" the lock fired late: the day closed at 105% of the limit. With "Every tick based on real ticks" it locked at 90.0% (−4,501), exactly as designed.

The reason: OHLC mode only knows 4 prices per minute. Inside a fast minute the equity can drop far below the level you check, and the tester never "sees" that drop until the next bar.

If you test anything that reacts to equity (loss limits, trailing equity stops, max drawdown guards), use real ticks or at least "Every tick". Otherwise the results look worse (or better) than reality.

Did anyone else see big differences between modelling modes for equity-based logic?
Youness Kanbaou:
Did anyone else see big differences between modelling modes for equity-based logic?

Of course we have seen that. Execution of SL's and TP's is estimated while testing on OHLC prices. This is not a bug. It's a feature.

Testing on OHLC prices is fine for EA's that trade on OHLC prices─in my Renko chart trading, for example.

Ryan L Johnson #:

Of course we have seen that. Execution of SL's and TP's is estimated while testing on OHLC prices. This is not a bug. It's a feature.

Testing on OHLC prices is fine for EA's that trade on OHLC prices─in my Renko chart trading, for example.

Agreed, for logic that decides on closed bars (Renko, bar-close entries) OHLC is fine and much faster.

The case where it bit me is the opposite one: a guard that has to react inside the bar. A daily loss lock watches equity tick by tick, and on OHLC it only gets 4 prices per minute, so on a fast candle it fires late. Entries can be tested on bars, but anything that protects the account from intra-bar moves needs ticks.

So my rule now: bars for the strategy, ticks for the risk layer.

Actual ticks can greatly affect strategies depending on the frequency and timeframes used.

Compare a mean reversion strategy on mean reversion (BB) with a time frame (TF) of 1 and the same strategy with a time frame (TF H1). Of course, in a market without a high spread like WIN$N, you'll see the utter chaos. Furthermore, depending on the historical data, it can even simulate synthetic ticks, and that's where the danger lies!

Samuel De Souza Ferreira #:

Actual ticks can greatly affect strategies depending on the frequency and timeframes used.

Compare a mean reversion strategy on mean reversion (BB) with a time frame (TF) of 1 and the same strategy with a time frame (TF H1). Of course, in a market without a high spread like WIN$N, you'll see the utter chaos. Furthermore, depending on the historical data, it can even simulate synthetic ticks, and that's where the danger lies!

Good point on synthetic ticks, that is the trap inside the "real ticks" option too. If the broker has no tick history for a period, the tester quietly fills it with generated ticks, and the result looks like a real-tick test when it is not.

What I check now before trusting any run: the "History Quality" line in the tester report. For a risk guard I want it close to 100% for the test days, otherwise the lock timing is again a guess.

And agreed on timeframes: the faster the strategy, the more the modelling mode matters. A risk layer is the extreme case, because it has to act within seconds whatever timeframe the strategy uses.
Youness Kanbaou:
I was testing a daily loss lock in the Strategy Tester (100,000 account, 5% daily limit = 5,000, lock set at 90% of the limit).

With "1 minute OHLC" the lock fired late: the day closed at 105% of the limit. With "Every tick based on real ticks" it locked at 90.0% (−4,501), exactly as designed.

The reason: OHLC mode only knows 4 prices per minute. Inside a fast minute the equity can drop far below the level you check, and the tester never "sees" that drop until the next bar.

If you test anything that reacts to equity (loss limits, trailing equity stops, max drawdown guards), use real ticks or at least "Every tick". Otherwise the results look worse (or better) than reality.

Did anyone else see big differences between modelling modes for equity-based logic?

It will always depends on the strategy. The higher the timeframe, the less frequent the trade, usually the more the M1 OHLC is closer to real ticks.
But basically the real test is using the real ticks option. I use M1 OHLC just to see if the strategy/setting is worth to look deeper as it's much faster than real ticks.

your test shows the lock reacts in time. What it can't show is the case where there are no ticks in between. If the market gaps at the open, the first tick after the gap is already past the limit, and real ticks or not, the lock fires there. I had that live on a prop account. So for me a lock at 90% means "starts closing at 90%", not "stops at 90%".

The second thing I'd look at is the close itself. In the tester everything closes at that one tick. Live, with several positions in a fast market, each one gets its own fill, so i'd expect the real number a bit worse than −4,501.

Thats why I like your point from the other thread about being flat before the weekend and big news. It covers the part no lock can.

Chris

the OHLC equity is interpolated, so intra-bar spikes never register. even on real ticks you'll need a buffer or check equity via OnTimer (1s) instead of OnTick. also 'every tick generated' behaves like OHLC for intra-bar; only 'every tick based on real ticks' is real

Erman Ardianto #:

It will always depends on the strategy. The higher the timeframe, the less frequent the trade, usually the more the M1 OHLC is closer to real ticks.
But basically the real test is using the real ticks option. I use M1 OHLC just to see if the strategy/setting is worth to look deeper as it's much faster than real ticks.
Agreed, that is a good workflow: M1 OHLC to filter ideas fast, real ticks for the final check.

The one exception I would add is anything that watches equity in real time. A strategy on H1 can be fine with OHLC because it only decides at bar close. But a loss limit or drawdown guard sitting on top of it has to react inside the bar, so for that layer only real ticks show the true behavior.

So my rule now: screen the strategy with OHLC, but test the risk guard with real ticks only.
Youness Kanbaou #:
Agreed, that is a good workflow: M1 OHLC to filter ideas fast, real ticks for the final check.

The one exception I would add is anything that watches equity in real time. A strategy on H1 can be fine with OHLC because it only decides at bar close. But a loss limit or drawdown guard sitting on top of it has to react inside the bar, so for that layer only real ticks show the true behavior.

So my rule now: screen the strategy with OHLC, but test the risk guard with real ticks only.
Yea that makes sense 

It would be useful to log equity at three points: when the limit is first detected, when each close request is sent, and after all positions are closed.

That would help separate detection timing from execution losses. I’d also compare the same real-tick test with no delay and a fixed execution delay, keeping all other settings unchanged.

Did your test use “No Delay”, or was an execution delay configured?

12
New comment