My EA passes the strategy tester but fails forward testing. - page 2

New comment
 
Hesham Ahmed Kamal Barakat #:
That tester/live gap is usually a mix of execution and data assumptions rather than a broken idea. I would compare three things first: spread at entry, slippage on exits, and whether the tick history matches the broker session you trade on. If the live account only breaks when spread widens or fills slip, the edge may be too sensitive to real costs. Have you logged the first 20 to 50 live fills against the tester fills to see where the divergence starts?
What exactly is a fill? 
 
wayneg1 #:
definately recomend monte carlo testing its so easy to over fit backfit strategies
How do you test an strategy with montecarlo?
 
Isaac Uriel Arenas Caldera #:
What exactly is a fill? 
A successful execution of a trade that returns filled status (result) from the trade server.
 
Ryan L Johnson #:
A successful execution of a trade that returns filled status (result) from the trade server.
thanks!
 
Getting a filled status is just the technical baseline. The real challenge is handling what happens immediately after that fill when the spread widens or slippage hits during high volatility. If your logic only checks for a filled result without accounting for latency and execution variance your backtest will never match your live account.
 
Vasile Ginu #:
Getting a filled status is just the technical baseline. The real challenge is handling what happens immediately after that fill when the spread widens or slippage hits during high volatility. If your logic only checks for a filled result without accounting for latency and execution variance your backtest will never match your live account.
I understand, perhaps a dynamic SL, or avoiding LTF...
 
Discrepancies between the Tester and Live environments are almost always due to asynchronous data or order validation. In the Tester, historical data is deterministic. Forward testing is subject to network latency, slippage, and requotes. Check if your EA depends on SymbolInfoDouble or OrderSend results; if you aren’t checking return codes and handling ERR_REQUOTE or ERR_OFF_QUOTES specifically for live accounts, your EA will ‘fail’ in forward tests while ‘winning’ in the tester.
 
Isaac Uriel Arenas Caldera:

The strategy that I am currently developing performs very well in the strategy tester, but when I do forward testing it fails.

How can I shorteh the gap between strategy tester data and forward testing?

This is a classic symptom of overfitting or lack of robustness testing. It is very common for an EA to perform well in the Strategy Tester but fail in forward testing, especially if it was optimized too heavily on historical data.

Robustness testing is essential to filter out strategies that are too sensitive to market changes. In StrategyQuant or similar platforms, this typically means running additional tests such as:

Monte Carlo simulations (to test randomness effects)

Multi-market validation (testing the same logic on different symbols or timeframes)

Walk-forward analysis (to verify stability over time)

If a strategy does not pass these tests, I recommend discarding it and continuing to generate new ones. With the right tools, you will find many that meet robust criteria. If you are not currently generating strategies that pass these robustness checks, I would suggest adjusting your approach—focus on out-of-sample testing and avoid curve-fitting.

The key is to accept that most strategies will fail, and the goal is to find the few that are genuinely robust. Keep testing, keep filtering, and you will eventually find the ones that work in real market conditions



 
Fernando Medina Villanueva #:
Monte Carlo simulations (to test randomness effects)
How do you do the Monte Carlo simulation? I haven't find a good way to do it...
 
Good answers above. Let me add three things we only learned by measuring them, because they tell you *where* the gap is coming from rather than just that it exists.

1. How much the tick model hurts you is proportional to how tight your exit is.

If your take-profit is a few dollars or pips away, modeled ticks will invent fills that never happened, and the backtest is fiction. We had a gold system that showed profit factor 2.46 on modeled ticks and 0.49 on real ticks over the same window with the same settings - the trade count barely changed, but the fills were phantom. Meanwhile a Donchian breakout with a stop 2 ATR away and no fixed take-profit gives nearly identical numbers on both models, because no decision depends on the intrabar path.

So: if your EA has a thin TP or scalps, assume the tester is lying until proven otherwise on real ticks. If it has wide ATR-based exits and the two models still disagree a lot, look for a bug (a peek at the forming bar is the usual culprit).

2. Test on a second broker's history. This is the cheapest overfitting detector there is.

Indicator-based systems in particular can live inside one feed's noise. We had strategies that produced profit factor 2.17 on one data source and 0.92 on another - identical code, identical settings, identical date range. That is not "different execution"; it means the edge was in the feed, not in the market. If your strategy survives a feed change with a recognisably similar profit factor, you have real evidence. If it collapses, you have just saved yourself a live account.

3. Some of the gap is not the strategy at all - it is the execution layer that the tester silently forgives.

Things that always pass in the tester and quietly fail live:
- Stop modifications that ignore SYMBOL_TRADE_STOPS_LEVEL and SYMBOL_TRADE_FREEZE_LEVEL, or that resend the same stop every tick. Live, these are rejected as invalid stops, so your trailing stop simply does not move.
- Modifying a stop while the session is closed (metals and indices have a daily break). Rejected live, ignored in the tester.
- Sending an order without checking OrderCalcMargin against free margin - live you get a rejection loop; in the tester it just fills.

Log every failed OrderSend / PositionModify with GetLastError() on your forward test. If your forward is worse than your backtest, the log often shows why in the first day.

One more thing on overfitting, since it was mentioned: after you optimize, plot the result against each parameter. A real edge shows a broad plateau - neighbouring values also work. A curve fit shows a lonely spike. Out-of-sample testing tells you *if* you overfit; the plateau test tells you *how close to the cliff* you are.

123
New comment