Curve-Fitted Backtests vs Real Edge — What’s the actual filter?
What I have seen is that the signals show the validity of a bot...
This is naive. There is no way to tell if the signal is actually running the EA as claimed. For example there is a signal of one of the top 3 EA's in the market where it is clear it is not the only EA running, if it is running at all. There are signals that appear running on a real acccount while it is clear it is a demo account. For example trading the news events with hardly any slippage.
Also, it is not curve fitted back tests that are a problem. It is hard-coded non trading periods in the EA to "beautify" back tests that are a problem. Always back test on a custom symbol with the time shifted. If the EA denies testing on a custom symbol (prohibited by the way according to the market rules) you know they are temperaing with the history.
What really separates one product from another in this marketplace when almost anyone can present a curve-fitted backtest?
If I had to reduce it to just two things traders themselves can do to filter out most of the nonsense, it would be this:
First, test the EA on shifted or different data.
Simply changing the time alignment or using a different data source will immediately expose strategies that rely on curve fitting or hardcoded conditions. A genuinely robust system should not fall apart because of a small shift in data.
Second, forward test it yourself.
Backtests can always be manipulated, but live or demo (live is better) execution over 100+ trades tells the real story. If the behavior diverges significantly from the backtest, that’s a major red flag.
Between those two, you can eliminate the majority of over-optimized or misleading systems.
I’ve been around MQL5 for quite a while, coding EAs, and I keep coming back to the same question:
What really separates one product from another in this marketplace when almost anyone can present a curve-fitted backtest?
What seems to be happening in practice is that presentation is rewarded more than actual robustness.
This creates an environment where well-presented systems can easily attract less technical users, who may not be able to distinguish between genuine robustness and over-optimized strategies.
The core issue, in my view, is that there is no real filtering mechanism that can distinguish a system with a genuine edge from one that is simply optimized on historical data.
Maybe it would make sense to introduce a second layer of evaluation beyond basic technical validation.
For example:
1). groups of experienced backtesters who can identify curve fitting.
2).evaluation from Live/Demo conditions like convergence. whether the behavior of the strategy remains consistent after runing lie for 100-200 trades.
3).or, more generally, a framework that emphasizes robustness rather than just tester results.
Because right now, distinguishing a genuinely robust system from a well-presented backtest remains difficult — even for experienced users.
mql5.com Signals cannot run in a demo account.
While a host of issues can affect the integrity of a Signal, a demo account is not one of them.
The core problem you are identifying is real. A backtest proves nothing about robustness on its own — it only proves the developer could find parameters that worked on historical data. The presentation layer has become more important than the substance layer and that is a genuine problem for buyers trying to make informed decisions.
The solution I built into my own system was to make the optimizer reject its own results. Every optimization cycle requires a minimum trade count to qualify, validates candidates against independent market periods the system has never trained on, and uses an inertial blending model so parameters never jump dramatically week to week. The goal was to make curve-fitting structurally difficult rather than relying on the developer's discipline to avoid it.
You can not have your 10 point break-even hit exactly the 10 points mark conistantly 2 minutes in NFP, it is just not possible
If I understood your post in another thread correctly, some CFD broker-dealers can allow limit orders to be placed at any price:
Forum on trading, automated trading systems and testing trading strategies
This is an inquiry regarding the reservation transaction system.
Enrique Dangeroux, 2026.03.29 11:10
It is in fact a limit of MT. There are brokers which allow to set an offset in order to place a buy limit above the current price or sell limit below.This could potentially limit a breakeven stop to a specific price.
Is such a broker-dealer allowance what you mean by?:
Obivousely the broker is in on it.
Or do you mean that the broker-dealer is conspiring with the Signal provider?
If I understood your post in another thread correctly, some CFD broker-dealers can allow limit orders to be placed at any price:
This could potentially limit a breakeven stop to a specific price.
Is such a broker-dealer allowance what you mean by?:
Or do you mean that the broker-dealer is conspiring with the Signal provider?
The strategy of the signal in question opens with a stop order and closes always with stop loss, both of which are market orders and thus subject to slippage, especially during high impact new events like NFP.
Willingly or not willingly, for sure, the broker accomodates this.
The strategy of the signal in question opens with a stop order and closes always with stop loss, both of which are market orders and thus subject to slippage, especially during high impact new events like NFP.
Willingly or not willingly, for sure, the broker accomodates this.
I guess that the Signal product name was removed from this thread before I happened upon it, but ok then.
If the broker-dealer is connected to the interbank FX market, this tends to make price stable (even over stable for scalping purposes)─especially for a pair like the Fiber (EURUSD). At least, this is the case for me in the "oddball" U.S.
Therefore, I wouldn't say "impossible" but maybe unlikely.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
I’ve been around MQL5 for quite a while, coding EAs, and I keep coming back to the same question:
What really separates one product from another in this marketplace when almost anyone can present a curve-fitted backtest?
What seems to be happening in practice is that presentation is rewarded more than actual robustness.
This creates an environment where well-presented systems can easily attract less technical users, who may not be able to distinguish between genuine robustness and over-optimized strategies.
The core issue, in my view, is that there is no real filtering mechanism that can distinguish a system with a genuine edge from one that is simply optimized on historical data.
Maybe it would make sense to introduce a second layer of evaluation beyond basic technical validation.
For example:
1). groups of experienced backtesters who can identify curve fitting.
2).evaluation from Live/Demo conditions like convergence. whether the behavior of the strategy remains consistent after runing lie for 100-200 trades.
3).or, more generally, a framework that emphasizes robustness rather than just tester results.
Because right now, distinguishing a genuinely robust system from a well-presented backtest remains difficult — even for experienced users.