Choosing a DRL Algorithm for MetaTrader: Why Drawdown Beats Return

Most DRL algorithm comparisons you find in the literature rank models by annualized return. If you plan to deploy the agent on a live MetaTrader account, that is the wrong column to sort on. Here is a benchmark that makes the point concrete.

The benchmark, read correctly

Across a set of commodity futures markets, four common DRL algorithms produced roughly these figures:

• DQN: 47.6% annualized return, Sharpe 0.81, max drawdown -16.6%
• PPO: 15.7% annualized return, Sharpe 2.04, max drawdown -0.75%
• A2C: Sharpe 2.11, max drawdown -5.2%
• DDPG: Sharpe 0.72, high drawdown

DQN's return is the largest in the table. Its drawdown is also the largest, and that is the number that matters. If you are running a prop firm evaluation with a 5-10% drawdown cap, a -16.6% peak-to-trough decline does not reduce your score — it ends the account. A2C looks tempting on Sharpe (2.11 versus PPO's 2.04), but its -5.2% drawdown is seven times worse than PPO's. The Sharpe advantage is real and irrelevant if the account is already gone.

Why DQN struggles on bar data

Two design properties explain DQN's drawdown, and both are structural rather than tuning problems.

First, the action space is discrete. Standard DQN chooses among fixed actions — buy, sell, hold. It decides whether to trade but not how much. On a 15-minute EURUSD strategy, position sizing is one of the main levers you want the agent to control, and DQN simply cannot express "buy 0.07 lots given current volatility."

Second, the max operator in the Bellman update causes overestimation bias. In a high-noise environment — which any 15-minute FX series is — the agent systematically overvalues actions that happened to sit next to reward spikes in training. Double DQN and Dueling DQN reduce the bias, and in NASDAQ tick-data market-making studies they were more stable than raw DQN, but neither removes the discrete-action limitation for bar-based trading.

Why PPO is the sensible default

PPO's clipped surrogate objective limits each policy update to roughly ±20% of the previous policy. In practical terms, a single volatile batch — an NFP week, a spike — cannot rewrite months of stable learning. That constraint is the mechanical reason PPO's drawdown is -0.75% in the benchmark while DQN's is -16.6%.

There is a second, less obvious reason to start with PPO: it fails diagnostically. When training curves oscillate and refuse to converge, the problem is almost always a reward-scaling issue or a lookahead feature, not the algorithm. Something worth remembering before reaching for a heavier model: a PPO agent on Kalman-filtered price data reached 27.1% CAGR on precious metals, while the same PPO on raw data managed 3.46%. The algorithm never changed. Input quality moved the result. Switching to SAC would not have fixed noisy inputs — it would only have made the noise harder to diagnose.

Enforcing the constraint on the MT5 side

Whatever algorithm produces the signals, the drawdown limit that decides survival is enforced on the MetaTrader side, not inside the Python agent. A minimal equity-drawdown guard in the EA keeps a bad regime from turning into a blown evaluation:

This is a safety net, not a strategy. The point of choosing PPO over DQN is that you rarely want this guard to fire in the first place — the algorithm's update dynamics keep you far from the cap rather than sprinting toward it.

The short version

Start new FX DRL projects on PPO. Run five seeds before drawing conclusions; if fewer than three produce positive out-of-sample Sharpe, fix the reward function and features before touching the algorithm. Move to SAC only when regime overfitting is confirmed or continuous sizing is a genuine requirement. For new bar-trading projects, DDPG and TD3 have no recommended starting role. And in every comparison, read the drawdown column first.