Discrepancy Between MT4 Strategy Tester Optimization Results and Backtest Outcomes

[Carl Emil Bograd]

Hello everyone,

I’ve been experimenting with my Expert Advisor in MT4’s Strategy Tester using the generic algorithm optimizer, and I’ve noticed a puzzling inconsistency: the parameter combination that delivers outstanding results during optimization almost flat-lines when I plug the same inputs into a standard backtest. I’d like to share my setup and observations in the hope that someone can shed light on what’s happening “under the hood” of MT4’s tester. This happens on both MetaTrader 4 Build 1440 and Build 1441, so it doesn’t seem tied to a single release.

---

Last week I ran a full optimization over the same GPYJPY data set (HST/FXT files downloaded from StrategyQuant), covering the exact same date range, using a fixed spread of 8 points and “Every tick” modeling. No visual mode, no code changes between runs—just pure Strategy Tester.

One of the standout runs (“Run 17”) delivered the following figures in the optimization report:

Run 17 → Net Profit: 28 605.51 USD  
Trades: 1 824  
Profit Factor: 1.73  
Expected Payoff: 15.68  
Maximal Drawdown: 6.91% 

Seeing over 28 600 USD profit on nearly 1 800 trades looked incredibly promising. However, when I immediately switched to a standard backtest with those exact same externals, the results collapsed to roughly 300 USD net profit on about 1 600 trades, with a drastically different equity curve (see attached screenshot).

I’ve double-checked that:

1. Identical data (same HST/FXT files) are loaded in both optimization and backtest.

2. Tester settings—date range, spread, “Every tick” model—are unchanged.

3. No calls to OnTester() or other hooks could be artificially boosting the optimized results.

4. Visual mode remains off throughout.

Despite all that, the optimized results simply don’t reproduce in the backtest. Has anyone seen behavior like this before? Are there known quirks in MT4’s generic optimizer—perhaps related to how it re-samples ticks, applies randomization within the spread, or caches data—that could explain the gap? Would generating custom FXT files with tick data from a separate source help ensure a one-to-one match? Or is there a logging or debug mode I’m missing that would reveal the exact tick sequence and execution times used during optimization?

Any insights, links to forum discussions, or best practices to guarantee fully reproducible strategy testing would be greatly appreciated. Thank you in advance for your time and expertise!

—

I’ve attached another screenshot below that illustrates an even more extreme case of the same problem. In this run the optimizer reported:

• Net Profit: 17 744.93 USD

• Trades: 558

• Profit Factor: 3.61

• Expected Payoff: 31.80

…but when I take those exact externals into a standalone backtest, I get only 1.51 USD net profit on 584 trades!

This is yet another clear example of how MT4’s generic-algorithm optimization can produce results that simply don’t reproducibly match a regular backtest, even though all data, modeling quality and settings remain identical.

—
Best regards,
Carl-Emil Bograd

[Michael Charles Schefe]

hence why I have not used mt4 strategy tester for long time. It took me years to get my mt4 strategy tester to work 98% same as my broker. I do not see the reason for your post. You are obviously highly aware of the same issue in the past, so why make a post about it now?

But i also did not use that online generator for very long as like all the other generators, is very misleading. I even got a 100% refund from my bank as i found the differences between data from that site, and my broker, and dukoscopy, where support claimed their data came from, was so different, they may have been from different universes.

NOTE: you are best to remove the name of the 3rd party product, otherwise this thread may be removed by a moderator, and you may get a warning.

[Miguel Angel Vico Alba]

Carl Emil Bograd:

Even if you're using the same FXT and HST files, MT4 might still simulate ticks a bit differently between an optimization and a regular backtest. That's because the "Every tick" mode doesn’t rely on real market data, it generates synthetic ticks from OHLC values, and that process isn't always consistent between test types.

If you're working with FXT and HST files from StrategyQuant, make sure MT4 doesn’t overwrite them. One way to do that is to launch the terminal in offline mode before running the backtest. Otherwise, MT4 might silently replace the files, and your backtest could end up using different tick data than the optimization did.

Also, depending on how your EA is written, especially if it uses static variables or relies on time-based logic, it might behave slightly differently during optimization runs.

These little differences may not seem like a big deal at first, but if your strategy depends on precise tick flow, like in scalping or timing-based entries, they can completely change the outcome over time.

Articles

Strategy Tester: Modes of Modeling during Testing

MetaQuotes, 2005.09.13 13:10

Note: Working with MT4 in 2025 is a bit like using Windows XP today; it still works, but it's outdated in many ways. If you rely on accurate and reproducible testing, you might want to seriously consider moving to MT5.

[Miguel Angel Vico Alba]

Michael Charles Schefe #: NOTE: you are best to remove the name of the 3rd party product, otherwise this thread may be removed by a moderator, and you may get a warning.

Regarding the mention of third-party tools, as long as there's context and no intent to promote, it's generally fine. The real issue is when there's clear advertising behind it.

Broker names, on the other hand, are not allowed under any circumstance. Just to clarify, this includes even neutral mentions; for example, saying that something was tested on a specific broker or works better with one is still not permitted, regardless of intent. These references are restricted to keep the forum neutral and avoid any appearance of bias or promotion.

In cases like StrategyQuant, if it's part of a technical discussion and not used for promotion, I personally have no problem with it. That said, if another moderator sees it differently, I’ll fully support their decision.