In-Sample Results Are Not Evidence: What We Learned by Killing Our Own AI Model

A model that has seen the data it is being scored on will look brilliant. That is not a discovery about markets; it is a description of memory. The only figures worth anything come from data the model never saw during fitting.

If a strategy's parameters — or a machine-learning model's weights — were chosen using a given period, then results measured on that same period are contaminated by construction. The fit was selected precisely because it scored well there. Reporting that score as performance is circular reasoning wearing a chart.

This is not only an AI problem. Running the Strategy Tester's optimiser over a period and publishing the best result is exactly the same mistake with a smaller vocabulary. The more parameters and the more combinations tried, the more certain it is that the winner is fitted to noise.

Which period were the parameters chosen on, and which period is the published figure from? If those two are the same, the number tells you nothing about the future. If the seller cannot answer, treat the answer as "the same".

The ONNX model shipped in Gold Neuron version 2.x was trained on data from January 2020 to July 2026. Its backtest inside that window was an in-sample fit and was never obtained in live trading. On out-of-sample years from 2016 to 2019 the model lost money in our tests, and the forward account showed negative expectancy. So we replaced the engine in version 3.10. We left the earlier backtest published, labelled as in-sample, rather than quietly deleting it.

An impressive in-sample curve is not weak evidence — it is no evidence. The honest move was not to defend the model but to retire it, and to state in the product description which window trained it.