Why Your DRL Agent Trades Fine in Backtest and Falls Apart Live: The Bridge Parity Problem
If you have trained a deep reinforcement learning agent in Python and tried to run it against a live MT5 terminal, you may have seen this: the agent converges cleanly, the backtest Sharpe looks good, and then on a paper account it behaves like it never trained. No error, no warning. Just quietly bad decisions.
In most of these cases the model is fine. The problem is the bridge — the layer that computes the observation vector from raw market data. If that computation differs even slightly between training and production, the agent receives inputs it never saw during training, and a policy is only as good as the distribution of inputs it was trained on.
The failure that produces no error message
Consider an example. The training pipeline fits Z-score normalization parameters over the full historical dataset and never saves them. The production bridge instead computes a rolling Z-score anchored to whenever the live system started. Same feature names, different mean, different standard deviation. Every normalized value shifts outside the training distribution, and the policy produces near-random actions with complete confidence.
The reason this is so hard to catch is that nothing throws. The tensor shapes match. OnnxRun() returns a valid action. The order goes out. The only symptom is performance that is worse than random.
Three parity failures worth checking first
- Normalization parameter drift. Freeze μ and σ during training, save them to a file, and load them in the EA. Recomputing them at runtime from live data defeats the entire purpose of normalization consistency.
- Position state encoding mismatch. Training encodes state as flat/long/short = {0, 1, 2}; the MQL5 bridge uses {-1, 0, 1}. The dimension matches, so the model accepts it — but "long" in production now has the numeric signature of "flat" in training. Directionally inverted decisions, full confidence.
- Session flags on the wrong clock. If the Python side uses datetime.now() with no timezone, it defaults to OS local time, while bar timestamps come back in the broker's server time. London/NY/Asian session flags land hours off, so session-conditional policies fire in the wrong sessions all day.
The bar-boundary trap
This one is specific to how MQL5 indexes bars. iClose(Symbol(), Period(), 0) is the close of the current, still-forming bar; it changes on every tick. iClose(Symbol(), Period(), 1) is the last completed bar. During training, every observation is built from completed bars only — the agent never once saw a partially-formed bar. If your live observation builder reads shift 0 for any price-derived feature, you are feeding the policy a value no training sample ever contained.
The distribution of the shift-0 close over a bar is genuinely different from the distribution of a completed close: higher intra-bar volatility, and in quiet regimes it sits biased toward the open. The fix is to gate the whole inference step on a new bar:
The same principle applies to every indicator you feed the agent. In MQL5, iRSI() returns a handle, not a value: create it once, then read the last completed bar with CopyBuffer() , not the current one:
Validate the bridge before any money moves
None of the checks below require live trading:
- Historical replay parity test. Take a 500-bar segment from the test period, run it through the production bridge, and log the full observation vector at every bar. Diff it element-by-element against the training pipeline output for the same bars. The maximum absolute difference should be under 1e-6 . Anything larger is a parity failure — find it feature by feature before proceeding.
- Random agent test. Before loading the trained policy, run a policy that picks buy/sell/flat uniformly at random on a paper account for a week. Cumulative reward should sit near zero, minus transaction costs. If it is badly negative, the environment or reward has a bias the random agent is exposing — the policy is not the cause.
- Canary account. Only then deploy the real policy on 5–10% of intended capital at minimum lot size for two weeks, and compare Sharpe and drawdown against the paper baseline.
Keep the guardrails out of the model
The policy maps observations to actions. It has no concept of "the Python process just crashed" or "this is an illiquid 3 AM session." Those belong in the EA: a hard stop-loss on every position, a hard position-size ceiling that overrides the agent's lot requests, a daily loss circuit breaker, and a heartbeat watchdog that treats a silent Python process as unavailable and stops opening positions. Mind the path the orders take: a Python order_send() goes from the terminal straight to the trade server, not through the EA, so either the agent sends its instructions to the EA and the EA places the orders, or the EA enforces these rules after the fill. The agent can be wrong about the market; it must never be able to leave an unmanaged position open during a connectivity failure.
Deploying a DRL agent to MetaTrader is an engineering problem, not a machine learning problem. Get the observation parity right, prove it against historical data, and enforce risk in the execution layer independently of the model. That covers the live deployment failures described here.
