What I Learned Chasing a Regime-Detection Signal (One Hypothesis Failed, One Held Up)

A few months into backtesting a trading model, I noticed something worth chasing: the model worked well in some stretches and poorly in others, and I wanted to know if there was a way to tell which kind of stretch was coming before it arrived. That question turned into a real research project, and it didn't go the way I first thought it would.

The first hypothesis: do multiple timeframes agreeing predict a good stretch?

The idea seemed reasonable: when several timeframes all point the same direction, maybe that agreement itself signals a more reliable trading environment. I tested it against about ten months of hourly gold data, split into 15 windows, and checked whether "timeframe agreement at the start of a window" correlated with how well the model actually performed during that window.

The result looked good - a real, moderate correlation. Encouraging enough that I wanted to confirm it on a second, independent dataset before trusting it.

It fell apart on a bigger sample

I pulled six more independent datasets - different timeframes, different multi-year spans, over a hundred additional data windows in total. The correlation that looked real on the first sample came back essentially as noise once averaged across all of them: the sign flipped constantly, sometimes positive, sometimes negative, with no consistent pattern at all.

This is worth sitting with for a second: a result that looks like a genuine pattern on one sample can be nothing more than noise once you test it properly. Fifteen data points agreeing with themselves isn't the same as a hundred data points across genuinely different periods agreeing with each other. I dropped the hypothesis entirely rather than try to rescue it.

The second hypothesis: does volatility regime matter?

A different idea, tested the same skeptical way: does how "normal" current volatility is (measured as current ATR relative to its recent average) say anything about whether the model's signals are likely to be reliable right now?

The first pass, at the same coarse "window level" as the failed hypothesis, looked weaker than I'd hoped - real but modest, and I almost filed it in the same bin as the first idea.

Testing it properly changed the answer

Instead of stopping there, I tested it at a much finer grain - individual bars, not aggregated windows - with a genuine chronological train/test split on half a million bars of data. Bucketed by volatility quintile, the pattern was clean and consistent on both the training data and the held-out test data: bars where volatility was close to its normal range showed meaningfully better model reliability than bars where volatility was unusually calm or unusually extreme in either direction.

Not "high volatility is bad" - abnormal volatility, in either direction, tends to mean less typical, less reliable conditions. That's a more specific and more interesting finding than the generic version, and it held up with real statistical weight behind it rather than a handful of coincidentally-agreeing samples.

What I built from this

SmartTrader Volatility Monitor surfaces this exact measurement for any symbol - live spread status, and a volatility reading that flags both "unusually calm" and "unusually volatile" as conditions worth a second look, not just the high side. It doesn't predict direction and doesn't recommend trades. It shows you a measured condition, the same one that took a failed hypothesis and a proper train/test split to actually validate, so you can decide what to do with it yourself.

Get it

Free on the Market: https://www.mql5.com/en/market/product/186906

From the same developer as SmartTrader AI Pro, built around the same principle: publish what held up, and be honest about what didn't. https://www.mql5.com/en/market/product/186191