We Trained a Neural Network for Gold, Then Deleted It: What the Out-of-Sample Years Showed

Version 2 of our XAUUSD system shipped with an ONNX neural model. Its backtest looked excellent, and we published it. Then we ran the test that matters and it failed.

The model was trained on data from January 2020 to July 2026. The impressive equity curve was measured inside that same period. That is not evidence of an edge; it is a demonstration of fit. A model that has already seen the data will describe it well whether or not it can predict anything.

We tested 2016 to 2019, which the model had never been shown. In our tests it was loss making. The forward account we were running showed negative expectancy over the same period of live observation. Two independent checks, both negative, on the only data that could have confirmed the model.

We had two options: keep publishing the flattering in-sample chart, or replace the engine. In version 3.10 the neural model was removed. The entry gate was rebuilt with gradient boosting and validated by walk-forward analysis, so the decision rule is always tested on periods it did not learn from. The update was free for existing users. The old backtest stays published on the product page, labelled as in-sample, because deleting it would hide the mistake rather than correct it.

Ask for two dates: the training window of the model, and the window of the backtest they are showing you. If those overlap, the number in front of you describes how well the model memorised the past. It says nothing about the future. This applies to our old version as much as to anyone else's current one.

The rebuilt system, with the full measurement history including the failure above: https://www.mql5.com/en/market/product/187329

All products and free utilities: https://www.mql5.com/en/users/app.develop.sk/seller