how much backtesting is enough for EA
let's say I've develloped an EA that i want to test.
On how many years should I test it to say that it can be used with confidence on a certain timeframe and currency pair ?
That depends on the type of strategy used and the time-frame it is most applicable on. There is no general rule! It can depend on many other factors too.
Exactly what it means.
It depends on the type of strategy used.
Does it heavily rely on tuning over historic data or does it make use of real time indicators, or does it only rely on time.
If it does rely on tunig over historic data then how long does the lookback period have to be.
Maybe it uses corelation then it will rely on linked values between economy's and backtesting will not have to be nessesary or even in some cases not possible.
Or it can use fixed values that only have to be gathered over historic data once.
It all depends on the strategy used, i only backtest to see if the logic fires correctly.
So it also depends on what you exactly want to back test for.
Personally i found that the things that work best, are the things that can not be tested, and the things that can be tested, usually fail on live market feeds.
Exactly what it means.
It depends on the type of strategy used.
Does it heavily rely on tuning over historic data or does it make use of real time indicators, or does it only rely on time.
If it does rely on tunig over historic data then how long does the lookback period have to be.
Maybe it uses corelation then it will rely on linked values between economy's and backtesting will not have to be nessesary or even in some cases not possible.
Or it can use fixed values that only have to be gathered over historic data once.
It all depends on the strategy used, i only backtest to see if the logic fires correctly.
So it also depends on what you exactly want to back test for.
Personally i found that the things that work best, are the things that can not be tested, and the things that can be tested, usually fail on live market feeds.
I agree with Marco.
Some strategies cannot be backtested at all because they use several EAs, some others cannot be backtested manually because they open multiple charts for multiple symbols.
The more I backtest is 4 to 6 weeks. All I verify is that the profit deviation is about the same along the weeks, in order to avoid over-optimisation.
Some say the EAs have to be backtested for years in order to prove they work, I'm ok with this but its not the only way. It all depends how the EA work. For instance, I notice that a breakout EA must be optimized weekly for a few weeks back. Parameters are quite different for july-august than for september-october. During the summer, almost all breakouts are wrong, so a TP must be set. In september, things start to happend and the TP can be suppressed, using a trail stop instead, because a few breakouts start in a trend.
It also mostly depends on your computing resources. Or financial, using the cloud. But backtesting a scalper for years won't help much. All that has to be determined is its black swan. Once a few big losses have been isolated, you can set this amount as stop loss and start a forward test in a demo account.
Also, my opinion is that EAs that show impressive backtests without any track record on a real account are to be avoided. I'm investing in a few Darwinex strategies, and those with less than one year stats are catastrophic.
So, as Marco, backtests are just to check if the strategy works, or determining a few constants. No need to backtest too long, but pick a few small periods here and there. Not the backtest, but the forward test on a real micro account will determine if the EA is okay, and this test must be more than one year. I'd say two or three.
- cloud.mql5.com
If you want to collect the best parameters for a specific symbol, or for all symbols, you can also concider the fact that you can do a simple and quick loop over all available bars in the OnInit() function, so then there won't be any reason to back test.
Then your EA will simply calculate best settings up to the current date when you attach it to a chart, and it will also update itself everytime you load it, or update on set intervals hardcoded into the code if you are not planning on removing it the first 10 years so you set it to update whenever market closes for example then it will reload on a weekly basis and stay up to date in comparison to the one time gathered ageing parameters found in the endless backtesting practise.
And mostly, the backtest is not to show that the EA performs well in the past, but to verify that the strategy will at least give the same results in the future.
For this, a long period is not needed, just apply two criterias. 1) Using option Forward=1/2, choose the result set that gives the smallest difference between forward and backward periods. 2) Using "hypothesis testing" (google it): Basically, multiply sharpe ratio by square root of number of days. If the result is more than 2.326, the profit is likely to be the same on next period. Of course the second method does not work well with a small sample of trades. It appears that this is not the duration of the backtest that is crucial, but the number of trades. Too few, the statistical method doesn't work, and too many gives averaged parameters that won't perform optimally. 2.326 corresponds to a 1% probability that the profit is zero with an infinite number of trades.
And mostly, the backtest is not to show that the EA performs well in the past, but to verify that the strategy will at least give the same results in the future.
For this, a long period is not needed, just apply two criterias. 1) Using option Forward=1/2, choose the result set that gives the smallest difference between forward and backward periods. 2) Using "hypothesis testing" (google it): Basically, multiply sharpe ratio by square root of number of days. If the result is more than 2.326, the profit is likely to be the same on next period. Of course the second method does not work well with a small sample of trades. It appears that this is not the duration of the backtest that is crucial, but the number of trades. Too few, the statistical method doesn't work, and too many gives averaged parameters that won't perform optimally. 2.326 corresponds to a 1% probability that the profit is zero with an infinite number of trades.
thanks Fred, I'm starting to get a grip now.
another question : what could be considered as a good sample of trades ? 100s ? 1000s ? 10000s ?
Strictly speaking, for an accurate statistic calculation, the theory says 1000 samples at least in any domain. But in reality you can get an acceptable approximation if the trading pattern is not varying much. Let's suppose you trade breakouts, the trades are opened after a ranging period, ended by two closing prices outside bolliger bands. The trade patterns are very similar so a lot less trades are needed. On opposite, imagine a strategy that opens random trades with a very good money management. In order to tune the MM parameters, you will need 1000 trades at least. Check your strategy and decide how the entry patterns are varying.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
let's say I've develloped an EA that i want to test.
On how many years should I test it to say that it can be used with confidence on a certain timeframe and currency pair ?