Market Regime Classification vs. Optimization Overfitting

[Gio Rendel Masagca Rivadillo]

I've been thinking about market regimes lately and I'm starting to question how we normally build EAs.

Most strategies are basically built around the assumption that the market behaviour we see in the backtest will continue to exist in the future. But we all know that isn't always the case.

A trend following system can perform really well during a strong directional market and then completely fall apart when volatility contracts and the market becomes more mean reverting.

So I was wondering if instead of constantly optimizing the parameters of an EA, it would make more sense to first identify what type of statistical regime the market is currently in.

There are several ways this could potentially be approached: volatility clustering, regime classification, hidden Markov models, Bayesian inference, unsupervised clustering, or even a relatively simple feature-based classifier using variables such as volatility, autocorrelation, momentum persistence, volume characteristics, and market structure.

The EA wouldn't necessarily need to predict the next price. It could just try to determine whether the current environment looks more like a trending regime, mean reverting regime, high volatility regime or something else.

Then the trading logic could change depending on the classification.

For example, during a trend regime the EA could use a trend following model, while during a mean reverting regime it could switch to a mean reversion model. During extreme volatility it could reduce exposure, and during an uncertain regime it could simply stay out of the market.

But there is something that bothers me about this approach.

If we optimize the regime detection using historical data, aren't we just introducing another layer of overfitting?

Instead of curve fitting the entry and exit rules, we might end up curve fitting our definition of the market regime itself.

So I'm not really sure whether adaptive EAs are actually more robust or whether we're just making curve fitting more sophisticated.

Has anyone here experimented with this seriously?

I'm interested in whether anyone has found a practical way to validate this kind of approach without relying too heavily on historical optimization. For example, how much weight do you put on walk forward testing, Monte Carlo analysis, and genuinely unseen out of sample data when deciding whether a regime based system is actually robust?

I'm also curious whether people think there is a point where an adaptive EA becomes too complex to trust. If the system is constantly classifying regimes, changing models and adjusting its behaviour, how do we know it is adapting to something meaningful rather than simply reacting to noise?

[fxsaber]

Gio Rendel Masagca Rivadillo:

If we optimize the regime detection using historical data, aren't we just introducing another layer of overfitting?

Here I proposed performing several optimizations within the optimization criterion itself, and then intersecting their results.

Unfortunately, the post is extremely difficult to understand.

[Gio Rendel Masagca Rivadillo]

fxsaber #:
Here I proposed performing several optimizations within the optimization criterion itself, and then intersecting their results.

Unfortunately, the post is extremely difficult to understand.

Thanks, I'll take a look into it.

[Jasiel Miranda]

The overfitting risk doesn't go away, it just moves up a level. Instead of tuning entries and exits, you end up tuning the number of regimes, the lookback windows, and the transition rules — same problem, more moving parts.

What I'd avoid is optimizing the classifier and the trading logic together against net profit. That pretty much guarantees the "regime" it finds is just whatever gives the best curve, not a real one.

Tested a range-oriented system on BTCUSDT a while back, and 11 of its first 15 positions were shorts during the Oct-Nov 2020 rally. The range filter kept giving the green light in the middle of a clear trend. Made me trust a lot less any classifier that isn't validated separately from the strategy it's feeding.

[Muhammad Fa'iq Abdi]

The biggest trap with regime-switching isn't just meta-overfitting—it's detection lag. Most statistical classifiers need a lookback window to confirm a state change. By the time the math confirms "we are now in a strong trend," the move is often already late-stage, so flipping your logic to a trend-follower usually means buying the top just before it rolls back into a range. Then, by the time it re-labels the market as mean-reverting, a breakout happens and you get chopped up on both sides.

In practice, the systems that actually survive out-of-sample don't use regimes to flip trading models 180 degrees. Instead, they use them purely as a risk governor: keep one core edge running, but cut position size in half or stay flat when volatility is in an extreme percentile or when the market structure is unclear. It's much safer to throttle exposure using a simple metric like rolling ATR percentiles than trying to tune complex classifiers that just end up curve-fitting historical noise.

[Valerii Sakara]

One category of bug that specifically shows up when people bolt a regime classifier onto an existing EA in MQL5: the classifier gets computed in OnTick() using CopyRates() over the last N bars including the still-forming bar (index 0). In the strategy tester that's fine because bars replay deterministically once closed — but on live/some brokers the lookback window silently includes a bar whose high/low/close are still moving, so the "regime" the EA reacts to on bar N can differ from what it would've detected once bar N actually closed. That's not curve-fitting in the walk-forward sense, it's leakage that only shows up as a live/backtest performance gap, and it's easy to miss because the backtest itself still looks clean.

Simple check: log the exact bar-close timestamp your classifier last used and diff it against the current bar's open time. If they match, you're reading a moving target.

Separately — +1 to validating the classifier on data the strategy tuning never touched. I've audited EAs where the "regime" boundaries were effectively memorized from the same in-sample window used to tune entries, which is the meta-overfitting Jasiel described — just hidden inside a feature-engineering step instead of a parameter.

[Ghenadie Tumco]

Gio Rendel Masagca Rivadillo:

From my own experience, I would answer: yes, adaptability can be seen as a kind of hidden optimization — but in practice, I have found the results to be very different.

For years, I tested thousands of strategies, especially through optimization. And again and again, I ended up with overfitting. Some strategies worked well for a certain period on live, but sooner or later they started to fail. Apart from being extremely time-consuming, optimization never gave me any real guarantee that the result would remain robust.

Because of that, I gradually moved away from optimizing EAs and started looking for mechanisms that adapt by themselves.

You can argue that this is still a form of optimization, just hidden inside another mechanism. I agree. But there is an important difference in what I have observed.

When I find an adaptive mechanism that performs well on its very first backtest, without searching for the best historical parameters, it tends to remain much more robust on out-of-sample data as well.

After that, I may do some fine tuning to improve profitability, but that is not the same as searching for a single perfect combination of parameters.

With traditional optimization, you often find a sharp peak surrounded by a valley — a result that looks excellent at one specific point, but deteriorates quickly around it.

With the adaptive mechanisms I have been working with, I am looking for something different: not a sharp peak, but a solid hill. The foundation itself already works reasonably well, and the tuning is only used to improve it.

So my answer would be:

Yes, adaptability can be considered a masked form of optimization. But in my experience, it can produce a much more robust result, because the foundation is a solid hill rather than a sharp peak surrounded by valleys.