A practical out-of-sample checklist for testing a trading robot
An attractive historical curve is a starting point for questions. The next useful question is whether the settings chosen during development still behave sensibly on a later period that did not influence their selection.
That is the purpose of an out-of-sample check. Here is a practical way to organize one in MetaTrader 5, with an explicit record of what was selected and when.
Define the split before choosing settings
An in-sample period is where you develop or select a configuration. The out-of-sample period is reserved for evaluating that frozen configuration.
For an illustrative chronological split, you could use January 2020 through December 2024 for development and January 2025 onward for evaluation. These dates are an example of a research design, not a claim about any product's results. Use enough relevant history for the instrument and strategy, and state the reason for the split.
Decide the symbol, timeframe, costs and intended review criteria before examining the reserved result. A split selected after browsing many attractive curves is a weaker test.
Freeze a reproducible package
Keep the compiled EA version, the exact input file and the test setup together. Record the broker server, symbol specification, deposit currency, leverage, position-sizing rule, tick model and date boundaries.
Include a file hash where possible. A filename such as “final.set” does not prove that the settings stayed the same. If you use the tester's forward optimization feature, distinguish its forward window from a separate holdout you have not inspected during selection.
The principle is straightforward: another person should be able to identify the same configuration and repeat the test without guessing what changed.
Run the later period with the same configuration
Do not optimize again on the reserved period and continue calling it an untouched evaluation. If its result leads you to change settings or logic, that period has become part of development. Describe the change and reserve new data for a later check.
Avoid silently changing position sizing between periods. Show absolute results alongside measures that help readers understand the exposure used. Keep trading costs consistent with the intended broker conditions and disclose simplifications.
Compare behavior as well as net profit
Review net profit after modeled costs, trade count, profit factor, drawdown, holding periods and concentration in a few trades or months. A positive result from a small number of trades deserves a different interpretation from a broad sample.
Ask whether the later period contains the situations the strategy was designed to face. Note major differences in volatility, spread or contract conditions. These observations explain the test; they are not reasons to keep retuning until the curve looks attractive.
Drawdown depends on exposure and position sizing. Report the risk settings and the path of losses so readers can understand what the curve represents. There is no single percentage that answers every trader's risk question.
Publish the two windows clearly
Show the in-sample and out-of-sample curves with visible date ranges, currency and risk settings. Label them as Strategy Tester simulations. Say how many configurations were considered and whether the later period remained untouched during selection.
If a report is missing, say so. If a test failed to run, that is an evidence gap, not a measured loss. A tester completion message alone is not a profitable result: inspect the actual report.
Out-of-sample testing reduces one source of optimism. It does not guarantee future performance, replace live execution evidence or remove trading risk. A clear record makes the evidence useful even when the result is disappointing.
Reference
MetaTrader 5 optimization and forward testing explains the platform's optimization and forward-testing workflow. The protocol above is a research checklist; it makes no performance claim about a particular EA.
