The problem of Optimization, and subsequent failure...
You need to learn what merits (financial and statistical) you ought to be using to evaluate your backtesting results (as you've noted the ones provided out of the box with Strategy Tester suck) such that backtesting becomes more a value-added tool for you rather than a "pie-in-the-sky hope generator".
https://www.mql5.com/en/articles/1413
https://www.mql5.com/en/articles/1526
https://www.mql5.com/en/articles/1530
http://www.futuresmag.com/Issues/2009/August2009/Pages/Minimizing-your-risk-of-ruin.aspx
From what I have passively gathered in observing posters and fellow traders over time is that the number one fallacy that most newbie traders (forex and other) fall into is the notion of stationarity. https://en.wikipedia.org/wiki/Stationary_process
That is, whether they realize they are or not, they make the false assumption that the statistical characteristics (average, stdev, drawdown, win-rate, etc) of their strategy is the one determined via backtesting and that those statistical characteristics ought to hold true going forward as well. (hence they believe the distribution of results to be stationary and that backtesting captures and reports this stationary distribution).
The second fallacy follows when the first comes true...that is the belief that the statistical characteristics are stationary but just not properly captured because there wasn't enough historical data in the backtest...hence the persistent request for more and more historical data.
Sounds like your over-optimizing your EA's. My advise would be: try to reduce the number of optimizable params as much as you can, by replacing those values by logic. Stoploss for example could be set by ATR or a factor of it. This would make your EA more adaptive to changes in the market, and less likely to be curve fitted.
hth
That is, whether they realize they are or not, they make the false assumption that the statistical characteristics (average, stdev, drawdown, win-rate, etc) of their strategy is the one determined via backtesting and that those statistical characteristics ought to hold true going forward as well. (hence they believe the distribution of results to be stationary and that backtesting captures and reports this stationary distribution).
The second fallacy follows when the first comes true...that is the belief that the statistical characteristics are stationary but just not properly captured because there wasn't enough historical data in the backtest...hence the persistent request for more and more historical data.
That's exactly my opinion too. Those are the two most common underlying (wrong) assumptions that most beginners have. The truth is u can't blame them; for most people these assumptions sound 'logical' and 'natural'. And as human nature goes, the moment they get positive results in a backtest, greed takes over logic and they get locked on to these assumptions. You can see this all over the forums... My recommendation to any beginner - don't optimize at all! Not a single parameter. Work on your strategy and then do a single pass only.
Regarding the default backtesting results template - I can't agree more - it does suck. It's not nearly enough to sufficiently evaluate the strategy's performance. It's a shame that MQ didn't make this part of the platform more flexible. I mean you can change the default template, but u can't define more calculations to add to that template... If they had done that then I am sure there would have been much more powerful templates developed by users and that even beginners would have used them. But as it is, it's much harder to do these calculations, hence most beginners only reach that level much too late (usually after losing a nice sum in the process).
There's another side to this, many of the more veteran traders eventually come to the opposite extreme and believe that backtesting is completely pointless and useless... It's a shame, cause if used and interpreted properly, it's a powerful tool!
That's exactly my opinion too. Those are the two most common underlying (wrong) assumptions that most beginners have. The truth is u can't blame them; for most people these assumptions sound 'logical' and 'natural'. And as human nature goes, the moment they get positive results in a backtest, greed takes over logic and they get locked on to these assumptions. You can see this all over the forums... My recommendation to any beginner - don't optimize at all! Not a single parameter. Work on your strategy and then do a single pass only.
Regarding the default backtesting results template - I can't agree more - it does suck. It's not nearly enough to sufficiently evaluate the strategy's performance. It's a shame that MQ didn't make this part of the platform more flexible. I mean you can change the default template, but u can't define more calculations to add to that template... If they had done that then I am sure there would have been much more powerful templates developed by users and that even beginners would have used them. But as it is, it's much harder to do these calculations, hence most beginners only reach that level much too late (usually after losing a nice sum in the process).
There's another side to this, many of the more veteran traders eventually come to the opposite extreme and believe that backtesting is completely pointless and useless... It's a shame, cause if used and interpreted properly, it's a powerful tool!
Well said!
Gordon do you know if the MT5 strategy tester improves on these deficiencies?
Gordon do you know if the MT5 strategy tester improves on these deficiencies?
You can do additional analysis from your strategy and print the results on deinit(). It slows down your backtest a bit, but it works for me.
I know.... I do that myself. But beginners have a hard enough time coping with coding EA's, they can't and won't add these kind of things to their EA... Which means they are stuck with the default Template.
If there was a facility to embed custom calculations into templates, then all they would have to do is download some more advanced templates from the codebase and copy them to their templates folder... That's something a beginner is much more likely to do. It would just make the platform much more flexible...
I'm very, very close to giving up this whole EA stuff, as it looks more and more like a terrible waste of time and effort... with no chance of success.
And just how long have you been trying?
I suggest you check out to see just how many EA driven systems are in profit..
Semper Fi
-BB-
You can do additional analysis from your strategy and print the results on deinit(). It slows down your backtest a bit, but it works for me.
If you are accumulating backtest output in an external file like csv then you can significantly reduce the time impact by creating a ram-drive and mounting the partition as your ...\tester\files\ folder. This gets around the issue of the limiting small-file IOPs of spindle drives. (and those j-micron controller, flash-based SSDs ;) )
This might be a well known approach but I figured I'd mention it on the off-chance it hadn't been openly discussed here before.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Hi, Anyone have some good advice on this problem ?
I've made several "good" EAs and they work well on BackTest, actually nearly any piece of cr*p can be optimized for good results when
backtested on the same piece of history as the optimization were done. But add another year to the beginning of the test and you may
(as I have) see that profit falls off, until it hits the point in history where optimization started... after which the profits may go straight up.
This is a sure sign that the EA only works on the period it was optimized, and that it will fail on forward and live trading.
Should there however be any good profit on forward / live it would just be pure coincidence and good luck... and it would be short lived!
So what can we do ? Are all EAs bound to fail ? Are there any special optimizing methods or procedures that work really well ?
I'm very, very close to giving up this whole EA stuff, as it looks more and more like a terrible waste of time and effort... with no chance of success.