Significant Performance Discrepancy Between Backtesting and Live Trading
Hello everyone,
I need to understand the source of the significant performance discrepancy I’m observing when testing an EA in backtesting versus live trading.
[edit]
Can anyone confirm if historical data is indeed processed in this way? Additionally, is there any reference to the algorithm used for data treatment? My goal is to attempt to replicate this processing in real-time before feeding the treated data into the EA's core strategy.
Any insights or guidance would be greatly appreciated.
Thanks!
Roberto
Hello
The lower the time horizon your ea trades then the more can be attributed to slippage and execution delays.
In all cases , whether you are on a short time horizon or a long one, you may have overfitted to the test period.
Also in all cases , if you optimized alongside management of the trades you have no data about the trades your strategy did not or could not have taken.
So you are dealing with a portion of the signal.
Even if you did not use the optimizer you may still have overfitted manually.
For example , if i am testing on a time period constantly ,this is it's real life equivalent :
In a dark room theres a table somewhere in the room and im holding an apple. My goal is to place the apple on the table.
If i keep training on the same room then i will learn where the table is in that room and not how to find a table in a dark room.
Hello
The lower the time horizon your ea trades then the more can be attributed to slippage and execution delays.
In all cases , whether you are on a short time horizon or a long one, you may have overfitted to the test period.
Also in all cases , if you optimized alongside management of the trades you have no data about the trades your strategy did not or could not have taken.
So you are dealing with a portion of the signal.
Even if you did not use the optimizer you may still have overfitted manually.
For example , if i am testing on a time period constantly ,this is it's real life equivalent :
In a dark room theres a table somewhere in the room and im holding an apple. My goal is to place the apple on the table.
If i keep training on the same room then i will learn where the table is in that room and not how to find a table in a dark room.
These are pretty accurate. It's either overfitting or order execution costs that makes a difference, although past performance does not guarantee future results.
One way I find very interesting to prevent overfitting is to slice time into different chunks. For example, I backtest on a 3-month period using different inputs and optimizations and then use the best, the average and the worst cases to backtest on the next 3-month.
Thank you, Lorentzos Roussos and Muhammad Azal, for your responses.
In my specific case, overfitting is not a concern—I am certain of this. I also acknowledge that slippage plays a significant role. However, I believe it alone cannot fully explain the substantial discrepancy between backtesting and live trading that I am observing.
Can you definitively rule out any form of preprocessing or filtering applied to historical data? Do you have specific knowledge to confirm this? If so, could you share any references or documentation on this topic?
I would really appreciate any insights you can provide. Thanks!
Thank you, Lorentzos Roussos and Muhammad Azal, for your responses.
In my specific case, overfitting is not a concern—I am certain of this. I also acknowledge that slippage plays a significant role. However, I believe it alone cannot fully explain the substantial discrepancy between backtesting and live trading that I am observing.
Can you definitively rule out any form of preprocessing or filtering applied to historical data? Do you have specific knowledge to confirm this? If so, could you share any references or documentation on this topic?
I would really appreciate any insights you can provide. Thanks!
From the words you mentioned, it seems the EA is a history reader.
It could be also overfitting issue but you would not notice it as soon as you place the EA on a real account.
How long have you been using it in a real account?
you said that you developed it, then post some code. That is how this forum works. Without seeing your code logic, then, we can only make guesses. Most posters that do not post code, do not get even the couple responses such as you have got.
Is it on mt4 or mt5? the strategy tester on mt4 is much more complicated to make a realistic scenario.
if your strategy uses pending orders, then, you may not have added stop levels or freeze levels. These have to be added in by coder.
I made half million demo dollars with a pending order ea, but when go live, the account went bust in a few days because I didnt realise this.
As I said, GUESSES.
Please consider which section is most appropriate — https://www.mql5.com/en/forum/172166/page6#comment_49114893
Thank you, Evren Caglar, Michael Charles Schef, and Fernando Carreiro.
I believe implementation details are not particularly relevant in this case—the strategy is quite straightforward, relying on immediate (OPEN, CLOSE) order placements. What truly matters is understanding whether the MT5 platform applies any preprocessing to historical data.
How can I verify this? Who should I contact to obtain a definitive answer? My broker has informed me that their MT5 platform simply relies on data provided by MQL and that they do not intervene in any way.
Any guidance on this would be greatly appreciated!
Thank you, Evren Caglar, Michael Charles Schef, and Fernando Carreiro.
I believe implementation details are not particularly relevant in this case—the strategy is quite straightforward, relying on immediate (OPEN, CLOSE) order placements. What truly matters is understanding whether the MT5 platform applies any preprocessing to historical data.
How can I verify this? Who should I contact to obtain a definitive answer? My broker has informed me that their MT5 platform simply relies on data provided by MQL and that they do not intervene in any way.
Any guidance on this would be greatly appreciated!
you have not even described how you setup your strategy tester. At least show an image of your settings that you have on the strategy tester, maybe someone can tell you better setting for your scenario.
Thank you, Evren Caglar, Michael Charles Schef, and Fernando Carreiro.
I believe implementation details are not particularly relevant in this case—the strategy is quite straightforward, relying on immediate (OPEN, CLOSE) order placements. What truly matters is understanding whether the MT5 platform applies any preprocessing to historical data.
How can I verify this? Who should I contact to obtain a definitive answer? My broker has informed me that their MT5 platform simply relies on data provided by MQL and that they do not intervene in any way.
Any guidance on this would be greatly appreciated!
A truly reputable and regulated broker provides their own data of high quality. They do not redirect it from MetaQuotes demo account data. If that is what your broker is doing, then they are not a reputable.
You were also not clear about which modelling methods you used in your back-testing. Was it real ticks or not?
Did you check how far back, your broker offers tick data? Are there any gaps in that data? Are there mismatches between tick data and the bar OHLC data?
Did you simulate delays in your testing to see your EAs robustness?
Did you test your EA's sensitivity to news events which results in high volatility, low liquidity, and widening spread?
All these and many more are the results of knowing the market conditions and your understanding of them. They are not a limitation of the Strategy Tester.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Hello everyone,
I need to understand the source of the significant performance discrepancy I’m observing when testing an EA in backtesting versus live trading.
I have extensively tested an EA (which I developed myself) using the MT5 platform provided by [edit] Markets. The backtesting results were excellent, which encouraged me to consider using it live. However, before committing real funds, I took the precaution of testing the EA on a demo account (from the same provider). In hindsight, this was a very wise decision, as the EA's live performance is nowhere near what I observed in backtesting.
[edit] Markets' support team attributes this difference entirely to slippage (the delay between order execution requests and actual execution), which is a plausible explanation. However, I find it hard to believe that slippage alone accounts for the drastic performance gap. I suspect that historical data undergoes some form of processing (such as filtering or noise reduction), which might make it easier for the EA to perform well during backtesting.
Can anyone confirm if historical data is indeed processed in this way? Additionally, is there any reference to the algorithm used for data treatment? My goal is to attempt to replicate this processing in real-time before feeding the treated data into the EA's core strategy.
Any insights or guidance would be greatly appreciated.
Thanks!
Roberto