Choosing a DRL Algorithm for MetaTrader: Why Drawdown Beats Return
Neural Networks

Choosing a DRL Algorithm for MetaTrader: Why Drawdown Beats Return

25 July 2026, 12:51
Boris Armenteros
Boris Armenteros
0
148

DRL algorithm comparisons often headline annualized return. If you plan to deploy the agent on a live MetaTrader account, that is the wrong column to sort on. Here is a benchmark that makes the point concrete.

The benchmark, read correctly

Wang and Liu (2025) compared four DRL agents on daily petroleum futures (Brent crude, plus RBOB and gasoil crack and time spreads), with a test period starting in January 2023. These are their Table 2 figures as printed. The paper gives no units, so we quote them that way:

  • DQN: annualized return 1.476, Sharpe 2.028, max drawdown 16.594
  • PPO: annualized return 1.450, Sharpe 2.041, max drawdown 0.752
  • A2C: annualized return 0.933, Sharpe 1.413, max drawdown 1.729
  • DDPG: annualized return 1.356, Sharpe 0.723, max drawdown 1.982

On return and Sharpe, DQN and PPO are indistinguishable. The drawdown column is where they split: 16.594 against 0.752, a 22-fold gap. The Calmar ratio, which sets return against maximum drawdown, says the same in one number: 1.977 for PPO, 0.088 for DQN.

Prop firm evaluations typically cap drawdown at 5-10%. Two agents with the same return and the same Sharpe can sit on opposite sides of that line, and only the drawdown column tells you which side. A2C had the second-lowest drawdown (2.3 times PPO's), but a lower Sharpe and the lowest return of the four.

One caution: this benchmark is petroleum futures over one test period, not FX. Read it as an illustration of why the drawdown column decides, not as a ranking to carry over to EURUSD.

Why DQN is a poor fit for bar-based FX

Two design properties count against DQN in bar-based FX trading, and both are structural rather than tuning problems. The benchmark above does not isolate them: every agent in it used discrete actions.

First, the action space is discrete. Standard DQN chooses among fixed actions — buy, sell, hold. It decides whether to trade but not how much. On a 15-minute EURUSD strategy, position sizing is one of the main levers you want the agent to control, and DQN simply cannot express "buy 0.07 lots given current volatility."

Second, the max operator in the Bellman update causes overestimation bias. In a high-noise environment — which any 15-minute FX series is — the agent tends to overvalue actions that happened to sit next to reward spikes in training. Double DQN and Dueling DQN reduce the bias, but neither removes the discrete-action limitation for bar-based trading.

Why PPO is the sensible default

PPO's clipped surrogate objective removes the incentive to change any action's probability by more than about 20% relative to the previous policy: the clip range ε = 0.2 of the original PPO paper (Schulman et al., 2017). In practical terms, a single volatile batch — an NFP week, a spike — is unlikely to rewrite months of stable learning. That is consistent with PPO's small drawdown in the benchmark, although the paper does not test clipping as its cause.

There is a second, less obvious reason to start with PPO: it fails diagnostically. When training curves oscillate and refuse to converge, look first at the reward scaling and for a lookahead feature, not at the algorithm. Input quality also has more leverage than algorithm choice. Kili et al. (2025) report that a PPO agent trained on Kalman-filtered gold prices (XAU/USD) reached a 27.1% CAGR, against 3.46% for PPO on the raw prices. The algorithm did not change; only the input did. Switching to SAC would not have fixed noisy inputs — it would only have made the noise harder to diagnose.

Enforcing the constraint on the MT5 side

Whatever algorithm produces the signals, the drawdown limit that decides survival is enforced on the MetaTrader side, not inside the Python agent. A minimal equity-drawdown guard in the EA keeps a bad regime from turning into a blown evaluation:

input double IN_MaxDrawdownPct = 8.0;   // hard equity drawdown cap (%)

double g_equity_peak = 0.0;

bool DrawdownBreached()
{
   const double equity = AccountInfoDouble(ACCOUNT_EQUITY);

   if(equity > g_equity_peak)
   {
      g_equity_peak = equity;
   }

   if(g_equity_peak <= 0.0)
   {
      return(false);   // divide-by-zero guard: no peak recorded yet
   }

   const double dd_pct = (g_equity_peak - equity) / g_equity_peak * 100.0;

   if(dd_pct >= IN_MaxDrawdownPct)
   {
      printf("%s: DRAWDOWN CAP HIT %.2f%% (peak %.2f, equity %.2f)",
             __FUNCTION__, dd_pct, g_equity_peak, equity);
      return(true);
   }

   return(false);
}

This is a safety net, not a strategy. Pick the algorithm by its drawdown column, and let this guard catch what the backtest did not show.

The short version

Start new FX DRL projects on PPO. Run five seeds before drawing conclusions; if fewer than three produce positive out-of-sample Sharpe, fix the reward function and features before touching the algorithm. Move to SAC only when regime overfitting is confirmed or continuous sizing is a genuine requirement. For new bar-trading projects, DDPG and TD3 have no recommended starting role. And in every comparison, read the drawdown column first.

Sources

Corrected in October 2026: an earlier version of this post misquoted the benchmark's figures. They are now quoted as printed in the paper, which changes the comparison: DQN and PPO matched on return and Sharpe ratio, and their drawdowns were 22 times apart.