Why Your DRL Agent Trades Fine in Backtest and Falls Apart Live: The Bridge Parity Problem
Neural Networks

Why Your DRL Agent Trades Fine in Backtest and Falls Apart Live: The Bridge Parity Problem

24 July 2026, 15:25
Boris Armenteros
Boris Armenteros
0
25

If you have trained a deep reinforcement learning agent in Python and tried to run it against a live MT5 terminal, you may have seen this: the agent converges cleanly, the backtest Sharpe looks good, and then on a paper account it behaves like it never trained. No error, no warning. Just quietly bad decisions.

In most of these cases the model is fine. The problem is the bridge — the layer that computes the observation vector from raw market data. If that computation differs even slightly between training and production, the agent receives inputs it never saw during training, and a policy is only as good as the distribution of inputs it was trained on.

The failure that produces no error message

One concrete example. The training pipeline fit Z-score normalization parameters over the full historical dataset. Those parameters were never saved. The production bridge instead computed a rolling Z-score anchored to whenever the live system started. Same feature names, different mean, different standard deviation. Every normalized value shifted outside the training distribution, and the policy produced near-random actions with complete confidence.

The reason this is so hard to catch is that nothing throws. The tensor shapes match. OnnxRun()  returns a valid action. The order goes out. The only symptom is performance that is worse than random.

Three parity failures worth checking first

  • Normalization parameter drift. Freeze μ and σ during training, save them to a file, and load them in the EA. Recomputing them at runtime from live data defeats the entire purpose of normalization consistency.
  • Position state encoding mismatch. Training encoded state as flat/long/short = {0, 1, 2}; the MQL5 bridge used {-1, 0, 1}. The dimension matches, so the model accepts it — but "long" in production now has the numeric signature of "flat" in training. Directionally inverted decisions, full confidence.
  • Session flags on the wrong clock. The Python side used datetime.now()  with no timezone and defaulted to OS local time, while bar timestamps came back in UTC. London/NY/Asian session flags landed six hours off, so session-conditional policies fired in the wrong sessions all day.

The bar-boundary trap

This one is specific to how MQL5 indexes bars. Close[0]  is the current, still-forming bar; it changes on every tick. Close[1]  is the last completed bar. During training, every observation is built from completed bars only — the agent never once saw a partially-formed bar. If your live observation builder reads Close[0]  for any price-derived feature, you are feeding the policy a value no training sample ever contained.

The distribution of Close[0]  over a bar is genuinely different from the distribution of a completed close: higher intra-bar volatility, and in quiet regimes it sits biased toward the open. The fix is to gate the whole inference step on a new bar:

static datetime lastBarTime = 0;
datetime currentBarTime = iTime(Symbol(), Period(), 0);
if(currentBarTime == lastBarTime)
   return;                 // no new bar closed — do nothing
lastBarTime = currentBarTime;
// Build observation from completed bars (index 1+) and query the model here

The same principle applies to every indicator you feed the agent. Read from the last completed bar, not the current one:

double rsi = iRSI(Symbol(), PERIOD_M15, 14, PRICE_CLOSE, 1); // index 1, not 0

Validate the bridge before any money moves

None of the checks below require live trading:

  1. Historical replay parity test. Take a 500-bar segment from the test period, run it through the production bridge, and log the full observation vector at every bar. Diff it element-by-element against the training pipeline output for the same bars. The maximum absolute difference should be under 1e-6 . Anything larger is a parity failure — find it feature by feature before proceeding.
  2. Random agent test. Before loading the trained policy, run a policy that picks buy/sell/flat uniformly at random on a paper account for a week. Cumulative reward should sit near zero, minus transaction costs. If it is badly negative, the environment or reward has a bias the random agent is exposing — the policy is not the cause.
  3. Canary account. Only then deploy the real policy on 5–10% of intended capital at minimum lot size for two weeks, and compare Sharpe and drawdown against the paper baseline.

Keep the guardrails out of the model

The policy maps observations to actions. It has no concept of "the Python process just crashed" or "this is an illiquid 3 AM session." Those belong in the EA: a hard stop-loss on every position, a hard position-size ceiling that overrides the agent's lot requests, a daily loss circuit breaker, and a heartbeat watchdog that treats a silent Python process as unavailable and stops opening positions. The agent can be wrong about the market; it must never be able to leave an unmanaged position open during a connectivity failure.

Deploying a DRL agent to MetaTrader is an engineering problem, not a machine learning problem. Get the observation parity right, prove it against historical data, and enforce risk in the execution layer independently of the model. That covers the majority of live deployment failures.

#machine learning, metatrader, MQL5, ONNX, deep reinforcement learning, DRL, live deployment