A ten-year backtest can serve two very different purposes. It can be the history used to develop and select an EA, or it can include periods reserved to assess decisions made earlier. The report may cover the same dates in both cases, but the evidence is not interchangeable.
That is the reason for separating Development, Validation and Final OOS. The split is useful only when it changes what the developer is allowed to do with each period. Three date labels attached to one continuous optimization process do not provide three independent checks.
Leave some data outside the development process
Using the entire history during development gives immediate feedback. A losing period suggests a filter; another weak patch suggests an entry adjustment. Retesting the full sample shows whether those changes improved that particular history.
What it cannot then provide is a clean answer to how the finished EA behaved on data that did not help shape it. This applies to candidate selection and portfolio construction as well as parameter tuning. Information enters the system whenever a result influences what is kept, changed or discarded.
A reserved period lets you check whether the candidate still met the relevant criteria after construction and selection were complete. That question becomes harder to answer if every available period has already been used to improve the result.
AI-generated illustration of data roles. These dates do not certify that every product component was unexposed to later-period data.
Give each period a defined role
The basic split discussed in this series is 2014–2022 for Development, 2023–2024 for Validation, and 2025 through the first half of 2026 for Final OOS. These are the roles in the framework, not proof that every component of a particular product was untouched by all later-period research.
Development is where hypotheses, rules and candidates are compared. It is appropriate to make changes in response to those results, because construction is the purpose of the period.
Validation examines whether the candidates retained from Development hold up in another interval. It may also inform narrowing or priority setting, provided that role is stated. If it does, its results should be described as part of selection, rather than as independent confirmation of the eventual winner. Repeatedly redesigning a candidate around Validation results makes it an extension of development.
Final OOS has a narrower job. The candidate or portfolio, relevant settings and acceptance criteria should be fixed before it is evaluated. The result is then assessed against those criteria, without changing the rules to make a disappointing outcome pass.
AI-generated illustration of the different roles of development, validation and final testing.
A final test can fail without needing to be rescued
Suppose the final result is weaker than expected. Adding another filter, choosing another EA or changing portfolio weights may improve it. But those changes have now used information from the reserved period. The next version is no longer being tested against a period that played no part in its construction.
This does not prohibit further research. It changes what can honestly be claimed about the next result. The same period can remain useful historical evidence, but it cannot become unused again because the revised EA looks better there.
There is also a distinction between rejecting a candidate under a predefined rule and adjusting that rule after the result is known. The former is the purpose of a final check. The latter turns its outcome into development feedback.
AI-generated comparison of using a final test for evaluation versus using it to guide further adjustments.
What the split can and cannot establish
A well-separated historical assessment still cannot guarantee future returns. Sample size, market regimes and execution assumptions continue to matter. Monte Carlo analysis, parameter variation, cost stress and portfolio-dependency checks investigate weaknesses that a date split alone does not address.
Nor does “Final OOS” tell you the full research history of the components. A portfolio combination may have been fixed before a final interval even though individual strategies had earlier exposure to that interval. Product-specific evidence needs to explain the level at which the separation applies.
When comparing EAs, ask which data was used to build the system, which data helped select it, and what was fixed before the final assessment. Then ask whether anything changed afterward. Those answers are more informative than the length of the backtest without that context.
Product details and testing conditions
For our EAs, read the product-specific period disclosures; this framework is not a claim that every component had an untouched holdout.
Specifications, published historical results and operating limits: EdgeDriven Gold Portfolio — XAUUSD · EdgeDriven Dollar Yen Portfolio — USDJPY.
The examples in this article explain evaluation methods; they are not test results for those products. Historical simulations do not guarantee future results. Leveraged trading can cause substantial losses.
