My EA's kill switch tripped after 15 cycles — should the backtest stop too?

New comment
 
Ran a frozen first version of a system on BTCUSDT perpetual. Still a Development run, not Validation. The halt happened during the first development segment, 1 Oct to 16 Nov 2020 UTC — later periods haven't been touched yet.

Before running it I'd set two rules: halt if cumulative P&L is negative after 15 completed cycles, and don't evaluate anything below 30 cycles. First rule fired before the second one could be satisfied.
  • Cycles completed: 15
  • Net result: -43.02 USDT
  • Return on 300 USDT: -14.34%
  • Profit factor: 0.473
  • Max drawdown: 23.38%
  • Win rate: 46.67%
Not just fees eating a small edge either — before modeled fees it was already down 34.96 USDT. The seven cycles closed by the trailing stop averaged +5.52 USDT. The eight closed by Layer 2 (a conditional exit that closes a losing position when at least two of three adverse conditions agree) averaged -10.20 USDT.

Here's the annoying part: as a capital-protection mechanism, the kill switch did exactly what it was supposed to. As an experiment, it cut things off before the 30-cycle minimum I'd set. So I can't honestly call this a pass or a fail — official result is NOT_EVALUABLE.

Questions for anyone who's dealt with this:
  1. Should a kill-switch rule be respected during backtesting too, even if it means never reaching an evaluable sample?
  2. Would it make sense to stop the official run but keep logging signals and counterfactual fills separately, in a shadow ledger?
  3. If I do that, does the shadow result count toward evaluating the strategy, or is it diagnostic only?
  4. How do you separate a rule meant to protect real capital from a rule meant to reach a predeclared minimum sample?
Leaning toward keeping the kill switch as official behavior and running the shadow continuation purely as a diagnostic — clearly labeled, not used to rewrite the original result.

New comment