Has anyone tried cross-validation for testing EAs?
Forward testing is very common when you want to train and test your strategy by dividing your data into two parts: training on one set and testing on the other.
Cross-validation, on the other hand, divides the data into equal parts (folds) and trains/tests the model across multiple iterations. Let's say we have 5 years (2020–2025). With cross-validation, we could train and test a model across multiple periods, for example:
- Train on 2021–2025, test on 2020
- Train on 2020 & 2022–2025, test on 2021
- Train on 2020–2021 & 2023–2025, test on 2022
- Train on 2020–2022 & 2024–2025, test on 2023
- Train on 2020–2023 & 2025, test on 2024
- Train on 2020–2024, test on 2025
This is just an example. We could also use different time periods and different numbers of partitions, allowing us to use all the available data for more robust validations.
Has anyone tried this?
What is it that you hope to gain by randomly decreasing your data sample size?
I simply backtest the full data sample on live account historic data, and then demo trade or micro live trade.
FYI, years 2020 through 2025 = 6 years.
What is it that you hope to gain by randomly decreasing your data sample size?
I simply backtest the full data sample on live account historic data, and then demo trade or micro live trade.
FYI, years 2020 through 2025 = 6 years.
More ways to validate an EA before exposure to real market conditions.
I mean, I want to investigate if there exists more ways to validate it before getting to market.
More ways to validate an EA before exposure to real market conditions.
I mean, I want to investigate if there exists more ways to validate it before getting to market.
Hmm...
An important principle of statistical analysis holds that lengthier sample sizes lead to more accurate test results.
As somewhat of a partial caveat, an important principle of pattern recognition holds that recent data should be weighted heavier than older data.
Therefore, maybe you could do:
- 2020 ─ 2025, with a weighted factor of 1,
- 2021 ─ 2025, with a weighted factor of 2,
- 2022 ─ 2025, with a weighted factor of 3,
- 2023 ─ 2025, with weighted factor of 4, and
- 2024 ─ 2025, with a weighted factor of 5.
You could average your test results manually, or code an MQL5 service to do it for you (this is not easy).
(The weighted factors above are merely generic examples here to illustrate application of the aforementioned principles).
Hmm...
An important principle of statistical analysis holds that lengthier sample sizes lead to more accurate test results.
As somewhat of a partial caveat, an important principle of pattern recognition holds that recent data should be weighted heavier than older data.
Therefore, maybe you could do:
- 2020 ─ 2025, with a weighted factor of 1,
- 2021 ─ 2025, with a weighted factor of 2,
- 2022 ─ 2025, with a weighted factor of 3,
- 2023 ─ 2025, with weighted factor of 4, and
- 2024 ─ 2025, with a weighted factor of 5.
You could average your test results manually, or code an MQL5 service to do it for you (this is not easy).
(The weighted factors above are merely generic examples here to illustrate application of the aforementioned principles).
I think that I follow...
For those examples which data would you use to train and test?
As I understand, 2020-2025 would be the test for an EA traineds ...-2020?
I think that I follow...
For those examples which data would you use to train and test?
As I understand, 2020-2025 would be the test for an EA traineds ...-2020?
Sorry. I was a bit obtuse regarding your OP. The Example in Post #3 is aimed at training only.
I'm really not a fan of forward testing timespans that are distant in history. Doing so would tend to "do violence" to the pattern recognition principle in Post #3.
I did some digging...
Take this with a grain of salt because I usually don't get trading info from Wikipedia:
Suppose you consider a strategy around a moving average. You take the first 3 months of data, and find that for that period a 20-minute moving average was optimal (using tick data). You then validate this rule by assessing its performance for the 4th month (i.e. profit, reward/risk or any other statistic of interest). Next, you repeat the optimization using data from month 2–4, and validate using month 5, and keep repeating this until you've reached the end of the data. The performance you get for the validation months (4-13) are your out-of-sample performance. (https://en.wikipedia.org/wiki/Walk_forward_optimization).
It seems to me that your EA Input parameters would have to be optimized every month during live trading to be aligned with the above example, if I'm understanding it correctly.
Furthermore, the article states that:
The evaluation of the trading strategy as a tradable asset is an entirely different process. (See the citations therein).
For more efficient solutions, see:
Forum on trading, automated trading systems and testing trading strategies
Sergey Golubev, 2026.06.23 19:33
There are many articles about walk-forward optimization for MT5 - you can find them in this search results' link for example: this one.
And there are some Market products (most of them are free); I quotes some of them (in description of some of the free product):
"... demonstrating how the built-in library WalkForwardOptimizer (WFO) for walk-forward optimization works ...": this one.
The author/coder of most of them is Stanislav Korotky (he is the famous coder and author of the MQL5 programming language book namely "MQL5 Programming for Traders" - https://www.mql5.com/en/book)
-----------------
So, no need to go to any external websites for this knowledge for example sorry.
I did some digging...
Take this with a grain of salt because I usually don't get trading info from Wikipedia:
Suppose you consider a strategy around a moving average. You take the first 3 months of data, and find that for that period a 20-minute moving average was optimal (using tick data). You then validate this rule by assessing its performance for the 4th month (i.e. profit, reward/risk or any other statistic of interest). Next, you repeat the optimization using data from month 2–4, and validate using month 5, and keep repeating this until you've reached the end of the data. The performance you get for the validation months (4-13) are your out-of-sample performance. (https://en.wikipedia.org/wiki/Walk_forward_optimization).
It seems to me that your EA Input parameters would have to be optimized every month during live trading to be aligned with the above example, if I'm understanding it correctly.
Furthermore, the article states that:
The evaluation of the trading strategy as a tradable asset is an entirely different process. (See the citations therein).
For more efficient solutions, see:
Sounds quite interesting, tbh, I read about what you mention and found that whaterver partion can be done, just need to be respect time, not training with the future and testing in the past.
I'll check the articles.
Btw, how do you do your forward testing?
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Forward testing is very common when you want to train and test your strategy by dividing your data into two parts: training on one set and testing on the other.
Cross-validation, on the other hand, divides the data into equal parts (folds) and trains/tests the model across multiple iterations. Let's say we have 5 years (2020–2025). With cross-validation, we could train and test a model across multiple periods, for example:
- Train on 2021–2025, test on 2020
- Train on 2020 & 2022–2025, test on 2021
- Train on 2020–2021 & 2023–2025, test on 2022
- Train on 2020–2022 & 2024–2025, test on 2023
- Train on 2020–2023 & 2025, test on 2024
- Train on 2020–2024, test on 2025
This is just an example. We could also use different time periods and different numbers of partitions, allowing us to use all the available data for more robust validations.
Has anyone tried this?