Curve-Fitted Backtests vs Real Edge — What’s the actual filter? - page 2

[Michael Charles Schefe]

Enrique Dangeroux #:

The strategy of the signal in question opens with a stop order and closes always with stop loss, both of which are market orders and thus subject to slippage, especially during high impact new events like NFP.

Willingly or not willingly, for sure, the broker accomodates this.

even during nfp a non commission account tends to be stable, ie no or only li'l slippage -- but if you are so much certain, then, you need to make complaint to service desk.

[Thomas Eduard Van Der Jagt]

David Tomblin #:

The core problem you are identifying is real. A backtest proves nothing about robustness on its own — it only proves the developer could find parameters that worked on historical data. The presentation layer has become more important than the substance layer and that is a genuine problem for buyers trying to make informed decisions.

The solution I built into my own system was to make the optimizer reject its own results. Every optimization cycle requires a minimum trade count to qualify, validates candidates against independent market periods the system has never trained on, and uses an inertial blending model so parameters never jump dramatically week to week. The goal was to make curve-fitting structurally difficult rather than relying on the developer's discipline to avoid it.

interesting approach, thankyou for your insight