ForumSections

How much does the tick path change your EA test?

New comment

I do most of my development runs on M1 OHLC because it's fast. Before I release anything I rerun the finished EA on real ticks. On EAs that wait for a setup and give the trade room, the two modes usually land close together. On anything with a tight target they can be far apart.

In 1 minute OHLC mode the Open, High, Low and Close prices are taken from real history. Only the arrival times of those four ticks inside the minute are generated, and a single spread is fixed for that bar. So the price levels are real, but the intra-minute path, the sequencing of the ticks and the spread are simulated. A short target can be reached by a generated tick in an order the real sequence would never produce. Real-tick mode lets the spread vary within the minute, so the two runs can diverge in which trade fills first. Switch to real ticks and the edge shrinks or flips. Live spread, slippage and latency add their own effect, and the tester can emulate network delay so you can measure that part too.

My own runs show both outcomes (MT5 Strategy Tester simulation, real ticks). The Pantheon EA finished at 17,544.08 USD on M1 OHLC and 16,235.41 USD on real ticks, so the edge dropped. The Crown EA came in lower on real ticks, 29,058.44 USD against 29,768.04 USD. These are my own Strategy Tester simulations, not live results, and the real-tick runs reported 57% history quality against 100% on M1 OHLC.

What I do now is check the tick coverage of the history first, via the History Quality figure in the report. Then I put the deal lists from both modes side by side and look for the first trade where they split. That tells me more than the summary numbers do.

If you've run a short-target EA both ways, where did the fills differ for you? I'd like to know which settings you used too, so someone else could repeat it.

If you want to try this yourself, take a free EA from the MQL5 Market, run its demo on M1 OHLC and again on real ticks, then compare the first diverging trade.

Shakir Mehmed:
If you've run a short-target EA both ways, where did the fills differ for you?
In my estimation, the differing net profits have more to do with the kind of trading EA being tested than the Tester. I routinely test on custom charts, and on real ticks and OHLC. My trading EA(s) always trade (in/out) on bar close, so any differences are negligible. I should note that I manually input my CME futures commissions into the Tester, and those commissions are the lion's share of costs incurred. Of course, that may not be the case for everyone.
When I saw that the tick path was changing drastically in one EA when changing risk size (percentage) and nothing else, then I knew something was wrong. My EA even held trades, so why such a difference? it's because different risk percentages changes everything, so then nothing should be hardcoded with point values. An EA should ideally work with percentages on everything so that it's working like one big algorithm where no conditions are hardcoded with arbitrary point values. Now when I change the risk percentage, I see the exact same tick path in different backtests, and that means things are coded properly.
For short-target EAs I’d look at three places where the tick path decides the result. First, bars where both SL and TP sit inside one M1 range: OHLC mode has to assume an order, real ticks don’t. Second, spread: OHLC modes take the spread stored with each M1 bar, which can be narrower than the spread at the moment of a real fill, and short targets feel that on every trade. Third, stop and freeze levels that reject a modification only on certain ticks. With 57% history quality, part of that 7% gap may simply be missing ticks rather than execution, so I’d check which days fall back to generated ticks before drawing conclusions. Then align both deal lists by entry time: same entry with a different exit usually points to intrabar ordering or spread at exit.
Conor Mcnamara #:
When I saw that the tick path was changing drastically in one EA when changing risk size (percentage) and nothing else, then I knew something was wrong. My EA even held trades, so why such a difference? it's because different risk percentages changes everything, so then nothing should be hardcoded with point values. An EA should ideally work with percentages on everything so that it's working like one big algorithm where no conditions are hardcoded with arbitrary point values. Now when I change the risk percentage, I see the exact same tick path in different backtests, and that means things are coded properly.

Good point (pun intended).😁

That becomes even more apparent in futures trading where a point is often merely a decimal value showing on the price scale of a chart, while tick size can be up to 5 times larger─and its tick value is really what we're getting at here.

Shakir Mehmed:

I do most of my development runs on M1 OHLC because it's fast. Before I release anything I rerun the finished EA on real ticks. On EAs that wait for a setup and give the trade room, the two modes usually land close together. On anything with a tight target they can be far apart.

In 1 minute OHLC mode the Open, High, Low and Close prices are taken from real history. Only the arrival times of those four ticks inside the minute are generated, and a single spread is fixed for that bar. So the price levels are real, but the intra-minute path, the sequencing of the ticks and the spread are simulated. A short target can be reached by a generated tick in an order the real sequence would never produce. Real-tick mode lets the spread vary within the minute, so the two runs can diverge in which trade fills first. Switch to real ticks and the edge shrinks or flips. Live spread, slippage and latency add their own effect, and the tester can emulate network delay so you can measure that part too.

My own runs show both outcomes (MT5 Strategy Tester simulation, real ticks). The Pantheon EA finished at 17,544.08 USD on M1 OHLC and 16,235.41 USD on real ticks, so the edge dropped. The Crown EA came in lower on real ticks, 29,058.44 USD against 29,768.04 USD. These are my own Strategy Tester simulations, not live results, and the real-tick runs reported 57% history quality against 100% on M1 OHLC.

What I do now is check the tick coverage of the history first, via the History Quality figure in the report. Then I put the deal lists from both modes side by side and look for the first trade where they split. That tells me more than the summary numbers do.

If you've run a short-target EA both ways, where did the fills differ for you? I'd like to know which settings you used too, so someone else could repeat it.

If you want to try this yourself, take a free EA from the MQL5 Market, run its demo on M1 OHLC and again on real ticks, then compare the first diverging trade.


Hello Shakir,

if I look at this as someone who buys EAs, the first thing I check in a seller's backtest is the modelling line in the report. "1 minute OHLC" or "Open prices only" on a scalper with a short target is something I don't trust much, for the reasons Eduardo listed. Same with a real ticks report with low history quality, because then part of it is generated ticks anyway.

When I'm not sure, I ask the seller for the same period on real ticks with the history quality shown. If that looks very different, thats already the answer.

Chris :)

My gap was larger than yours because my entries trigger inside the bar. The same rules run as real EA code on real ticks earned a third to a fifth per trade of what my bar based engine had shown. On one breakout setup the engine showed 0.72 R per trade over 250 days where the real tick run came out at zero. Almost half of those triggers fell in the first minute of a move, so both sides of the bracket sat inside one M1 bar and the assumed bar path picked the direction. The two modes agreed on the direction in only 78 percent of those trades.

What located the fill assumptions for me was a third run on data with no edge in it. I took real M1 history and flipped each bar around its open at random, so nothing is left to predict. The honest result there is about zero before costs, yet the bar engine still showed a profit, which pointed straight at fills booked at the level while the trigger minute had already opened past it. On the 57 percent quality: on my broker's demo server real ticks start in spring 2023, and before that even real tick mode runs on ticks generated from M1 bars, so I compare the modes only from that date on.

Shakir Mehmed:

I do most of my development runs on M1 OHLC because it's fast. Before I release anything I rerun the finished EA on real ticks. On EAs that wait for a setup and give the trade room, the two modes usually land close together. On anything with a tight target they can be far apart.

In 1 minute OHLC mode the Open, High, Low and Close prices are taken from real history. Only the arrival times of those four ticks inside the minute are generated, and a single spread is fixed for that bar. So the price levels are real, but the intra-minute path, the sequencing of the ticks and the spread are simulated. A short target can be reached by a generated tick in an order the real sequence would never produce. Real-tick mode lets the spread vary within the minute, so the two runs can diverge in which trade fills first. Switch to real ticks and the edge shrinks or flips. Live spread, slippage and latency add their own effect, and the tester can emulate network delay so you can measure that part too.

My own runs show both outcomes (MT5 Strategy Tester simulation, real ticks). The Pantheon EA finished at 17,544.08 USD on M1 OHLC and 16,235.41 USD on real ticks, so the edge dropped. The Crown EA came in lower on real ticks, 29,058.44 USD against 29,768.04 USD. These are my own Strategy Tester simulations, not live results, and the real-tick runs reported 57% history quality against 100% on M1 OHLC.

What I do now is check the tick coverage of the history first, via the History Quality figure in the report. Then I put the deal lists from both modes side by side and look for the first trade where they split. That tells me more than the summary numbers do.

If you've run a short-target EA both ways, where did the fills differ for you? I'd like to know which settings you used too, so someone else could repeat it.

If you want to try this yourself, take a free EA from the MQL5 Market, run its demo on M1 OHLC and again on real ticks, then compare the first diverging trade.

One point that created the most significant difference in my comparison: according to the MT5 documentation, in "1 minute OHLC" and "Open prices only" modes, stop-loss, take-profit, and pending orders are executed exactly at the price you specified. In "real ticks" mode, however, they are executed based on the Bid/Ask prices at that specific moment. Consequently, if the market gaps over your stop-loss level (for instance, over the weekend), the OHLC mode will not reflect any additional cost, whereas the "real ticks" mode will.

A helpful approach: separately identify trades where both the stop-loss and take-profit levels fall within the same one-minute candle, as well as trades that were closed due to a market gap. If these two groups account for a large portion of your total trades, the results from the OHLC mode reflect the tester's assumptions more than the strategy itself. If they represent only a small fraction, then the OHLC mode is sufficient for an initial screening of the strategy.
A.Kian #:
One point that created the most significant difference in my comparison: according to the MT5 documentation, in "1 minute OHLC" and "Open prices only" modes, stop-loss, take-profit, and pending orders are executed exactly at the price you specified. In "real ticks" mode, however, they are executed based on the Bid/Ask prices at that specific moment. Consequently, if the market gaps over your stop-loss level (for instance, over the weekend), the OHLC mode will not reflect any additional cost, whereas the "real ticks" mode will.

My solution to the real ticks/OHLC prices dilemma is not to choose one of the two, but to deliberately design my EA's based on compatibility with both modes of the Tester. My EA's trade exclusively on custom charts and on bar/brick close prices. For custom charts, my first step is to run a "dummy test" on the underlying tradable symbol for the entire duration of my intended backtest of the custom symbol─just to load real tick data into the Tester. For that first run, I set my trading time filter to something absurd like allowing only 10 minutes of trading per day. That speeds up the run substantially. Again, the actual backtest results are irrelevant.

After that run completes, I run a 2nd test on the custom symbol/chart on real ticks. My CME Futures broker excludes commissions from MT5, so net profit from that 2nd run is merely "single net" profit. Manual commission settings have no effect in real ticks mode. Then I run a 3rd test on OHLC prices, keeping all other settings the same as in the 2nd test─except for my manually input commission settings which take effect in OHLC prices mode. The profit from that 3rd run is "net net" profit. Finally, I use the difference between the profits of the 2nd test and the 3rd test, and the total number of deals from the 2nd and the 3rd test, to "rough verify" the total amount of commissions. I say "rough" because floating spreads are excluded from testing on OHLC prices, and I usually input a pretty hefty spread in OHLC prices mode.

There is generally little to no material difference between the two testing modes when I use this method. I should probably note that these are scalping EA's (not HFT) running/trading on top-of-book data (not level 2 nor 3).

Eduardo and A.Kian already covered the main case: when stop and target can both be touched inside the same M1 bar, the OHLC model has to guess the order.

Two more checks before trusting the real-tick run itself. First, check History Quality per month and cut the comparison at the date your broker's real ticks actually start; below that, even "real ticks" mode is partly generated. Second, rerun once with a random execution delay. If the result only holds with zero delay, the difference is in the fill, not in the rules. Tester results are hypothetical and do not predict live trading.
Comparing the two reports' totals hides where the gap comes from, so compare them trade by trade. On the Backtest tab, export both reports and match deals by symbol, direction and entry time within one M1 bar. Then sort the gap into three groups: trades that exist in only one run, matched trades with a different exit price, and matched trades with the same exit but different costs. With tight targets, the second group usually explains most of it. Check those trades for SL and TP both inside one M1 range. If most of the gap is in the first group, the entry logic depends on intrabar ticks, and the real-tick run is the only one to trust. Also restrict both runs to months where History Quality is close to 100%, or you're partly comparing generated ticks with generated ticks.
12
New comment