Discussing the article: "Offline Reinforcement Learning in MQL5: Conservative Q-Learning from Historical Trade Logs Without Live Exploration"

New comment

Check out the new article: Offline Reinforcement Learning in MQL5: Conservative Q-Learning from Historical Trade Logs Without Live Exploration.

Build and deploy a conservative, offline‑trained policy entirely in MQL5. Logged trades become a fixed dataset for a two‑layer Q‑network trained with Conservative Q‑Learning, which pushes down Q‑values for poorly supported actions to limit extrapolation error. The EA loads the frozen weights and applies an out‑of‑distribution guard using saved feature statistics. This yields reproducible training and execution without live exploration.

Most reinforcement learning approaches to trading assume the agent can interact with the market during training — it tries an action, observes a reward, and updates. That's reasonable in a simulator; dangerous on a live account, since "trying an action" means a real order with real slippage while the policy is still bad at its job. Training offline removes that risk, but not the risk of deploying a bad policy once training is done: a frozen policy can still trade badly, it just can't make things worse by continuing to experiment.

This article works through a different setup: training a trading policy entirely from a fixed batch of historical transitions, with no further market interaction during training. The dataset is whatever a rule-based strategy, a manual trader, or a prior EA already logged — states, the action taken, the reward realized, and the state that followed. The policy learns from that log alone, then deploys frozen: no exploration, no live trial-and-error, no chance for learning itself to cause money loss.

Offline training also matters for auditability: a fixed dataset can be scrutinized and replayed before any order is placed, with every transition inspectable in a spreadsheet — worth the trade for a desk that has to explain its systems to a risk committee, even though an online agent's exploration policy keeps changing. This article doesn't demonstrate formal dataset versioning, a seed/parameter log, or an explicit train/test split, though — call it replayable rather than fully audited.

Author: Adedayo David Gbadebo

New comment