Backtesting Results vs Reality.
Hi All,
Recently finished optimising an EA that performed really well on back testing. Didn't report a single loss.
On it's first day of running today however it has drastically failed and has been quite painful to watch.
The spread was set correctly on back testing as there is a fixed spread on the DAX during the London session so I'm just not sure what to think.
Just wondering if anyone has anything to say about how this may have occurred?
Just don't rely always on the results of your backtests. Do forward and backtesting, before adjusting your code, it will help you a lot in finding bugs whatsoever. Especially market is changing, due to political news etc etc etc..
Hi All,
Recently finished optimising an EA that performed really well on back testing. Didn't report a single loss.
On it's first day of running today however it has drastically failed and has been quite painful to watch.
The spread was set correctly on back testing as there is a fixed spread on the DAX during the London session so I'm just not sure what to think.
Just wondering if anyone has anything to say about how this may have occurred?
Well, even if the spread is constant, you still have a problem with volume from broker during a live test.
In my opinion, you can't use a backtest to check for profit. It's only for a programmer to check whether their code is working or not.
If you want to see real results, a live account is the best choice. Either demo or real account should be fine.
Good luck~
Hi All,
Recently finished optimising an EA that performed really well on back testing. Didn't report a single loss.
On it's first day of running today however it has drastically failed and has been quite painful to watch.
The spread was set correctly on back testing as there is a fixed spread on the DAX during the London session so I'm just not sure what to think.
Just wondering if anyone has anything to say about how this may have occurred?
Like that one ?
Forum on trading, automated trading systems and testing trading strategies
Hedge saved you, who will save Hedge ?
Icham Aidibe, 2018.02.24 21:41
The same draft one week later.
EDIT : >300Go datas, hehe
"Not a single loss", is a good sign of unfeasibility : spread, execution delay, stop/freezelevels. I check it by converting the code into mql4 when I used mql5 at first, or mql5 when I used mql4 at first - it should work on the two :)
It's mine, here's the mql5 curve :
https://www.mql5.com/en/blogs/post/715169
If you're very doubtful, you can convert it in another language ... you're ready to go live when results are unanimous.
- 2018.01.19
- Icham Aidibe
- www.mql5.com
The summary: All (not yet) about Strategy Tester, Optimization and Cloud
including the following:
- Testing trading strategies on real ticks and the explanation is on this post.
Forum on trading, automated trading systems and testing trading strategies
Sergey Golubev, 2017.02.15 06:02
You can read this article (TESTING TRADING STRATEGIES ON REAL TICKS).
-------------------
Just an example -
MT4 and very old builds of MT5: I optimized the EAs from this thread just to find good settings for the pair, I was backtesting them, and I traded on demo for some time just to be sure that EAs are profitable.
With new builds of MT5: I will optimize the EAs from this thread to find the settings, and I will backtesting them with 'every tick based on real ticks' - and it will be same as trading by those EA on real account for many months or years!
Forum on trading, automated trading systems and testing trading strategies
Sergey Golubev, 2017.02.17 20:53
If you are backtesting EA on MT5 using 'every tick based on real ticks' so it will be almost same with trading on MT5 platform with some particular broker (because it is based on actual historical data).
Example, read this thread: Why is it better MT5 than MT4?? Does it have fewer limitations ??? - this is the quote from the first post of the thread:
- In MT5 you can backtesting robots with the closest possible conditions to the real market natively (real tick data, real variable spreads, lag, slippage, etc). In MT4 you can't natively. You only can if you pay for a third-party software. If so, you also have to download history data from a few sources (there are many few, almost everyone uses the same source), transform it to MT4 format and open the platform through this third-party software in order to patch MT4 behavior. You take many hours to complete this process, and you have to repeat it every time you want to incorporate new data.
We have all seen hundreds of robots that obtained spectacular results in backtesting, but when operating in real account the results were very bad. This is mainly because they were made with conditions that had nothing to do with real market conditions.
For more information about it - read this summary.
--------------
As i know - some coders/traders are converting their MT4 EAs to MT5 just to backtest them and/or to find the settings with optimization to get the backtesting results that are closest to reality.
Hi All,
Recently finished optimising an EA that performed really well on back testing. Didn't report a single loss.
On it's first day of running today however it has drastically failed and has been quite painful to watch.
The spread was set correctly on back testing as there is a fixed spread on the DAX during the London session so I'm just not sure what to think.
Just wondering if anyone has anything to say about how this may have occurred?
This occurred because you made a classical error assuming that a back test result would have any relationship to real market performance.
The other way around is also possible, a robot that fails miserably on the back tester, but makes Millions on a real market feed.
So the best way to deal with this could be to lower your expectations or discard the tester altogether and start to work with real results from the start with smaller position sizes.
It might be not as fast but surely the results you get are trustworthy.Hi All,
Recently finished optimising an EA that performed really well on back testing. Didn't report a single loss.
On it's first day of running today however it has drastically failed and has been quite painful to watch.
The spread was set correctly on back testing as there is a fixed spread on the DAX during the London session so I'm just not sure what to think.
Just wondering if anyone has anything to say about how this may have occurred?
This occurred because you made a classical error assuming that a back test result would have any relationship to real market performance.
The other way around is also possible, a robot that fails miserably on the back tester, but makes Millions on a real market feed.
So the best way to deal with this could be to lower your expectations or discard the tester altogether and start to work with real results from the start with smaller position sizes.
It might be not as fast but surely the results you get are trustworthy.Anticipating. No the tester isn't creepy. Let me give you an example.
In netting mode, positions with higher size than maxlot counts for 1 position. Let's say : 2*100 lots = 200 lots = 1 position.
When the tester closes it, it does it cleanly, once.
In real conditions, there's a delay between each 100 lots closed, a delay being a price difference & a profit difference.
That kind of "things" rather than spread, or datas, which can change everything.
Of course you won't set up your EA on real feed prices, too fastidious ... but you have to consider real conditions as much as possible, in your code, while testing - note that it supposes to also consider server disconnection which is a reality.
The tester is not a real market feed.
I would compare the two by crashing a flight simulator into the ground and then crashing a real plane into the ground, and look at the differences between the two events.
To assume that the two could possibly result in the same outcome is the very wrong perspective to start from.
And many people make this mistake they think that if it runs successfully in the tester it will also on a real feed, but as you can see time and time again it does not.
It's all part of the learning process.The tester is not a real market feed.
I would compare the two by crashing a flight simulator into the ground and then crashing a real plane into the ground, and look at the differences between the two events.
To assume that the two could possibly result in the same outcome is the very wrong perspective to start from.
And many people make this mistake they think that if it runs successfully in the tester it will also on a real feed, but as you can see time and time again it does not.
It's all part of the learning process.Sure it's not but ...
Imagine Airbus or Boeing or any else, building a plane, trying to take off, crashing, rebuilding, re-testing, re-crashing ... what outcome ?
The flight simulator is unavoidable, it means to simulate as close as possible flight's real conditions hoping that it's close enough to make the flight possible and successful at the first try.
Otherwise, i'm afraid to imagine the extra-cost relative in plane industry :-/
To come back to our tester, from MT4 to MT5, it has been improved. In MT5 it has been improved too, I'm thinking of execution delay option. The real tick mode has been introduced too, for now it's obviously yet not perfect, but efforts are made in this direction, efforts that augur much more accuracy in the future.
The simulator is a means for pilots to practice flying.
If the tester was a means to teach traders trade, then , yes.
But this flight simulator is mostly used to fly on autopilot.
The simulator is a means for pilots to practice flying.
If the tester was a means to teach traders trade, then , yes.
But this flight simulator is mostly used to fly on autopilot.
Come on ! You just want the last word ...
https://youtu.be/7rMgpExA4kM?t=82
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Hi All,
Recently finished optimising an EA that performed really well on back testing. Didn't report a single loss.
On it's first day of running today however it has drastically failed and has been quite painful to watch.
The spread was set correctly on back testing as there is a fixed spread on the DAX during the London session so I'm just not sure what to think.
Just wondering if anyone has anything to say about how this may have occurred?