Discussing the article: "Behavioural Cloning in a Live Trading System: What Live Experience Can and Cannot Teach"

New comment

Check out the new article: Behavioural Cloning in a Live Trading System: What Live Experience Can and Cannot Teach.

This article measures a live-to-train behavioral-cloning loop for an RL trading system across 83 models. Outcome-weighted imitation shifted the policy toward Hold (48% → 82%), with 12 final models not trading at all. Live records were instead used to fit spread and slippage to broker fills, revealing execution costs ~85× higher than in simulation. We describe the loop, the failure mechanism, the calibration, and a protocol to apply.

A reinforcement-learning trading system typically has two parts: a trainer that learns from historical bars and a live account that executes real trades. The obvious idea — obvious enough that I built it — is to feed the second back into the first. In reinforcement learning (RL) this is called behavioural cloning (BC), which makes the policy imitate demonstrated actions. If the demonstrations are the system's own profitable live trades, the loop seems self-improving. Trade, learn from what worked, trade better.

In my previous article, Reinforcement Learning Meets MetaTrader 5, I described the pipeline around a PPO agent on XAUUSD M15 and reported the honest result: after 763 live demo trades, a profit factor of 1.05 and a t-statistic of 0.55, indistinguishable from zero. That pipeline contained a behavioural-cloning loop from the start. I did not write about it then because I had not measured it. This article is that measurement.

The short version: the loop ran for two months, and over that time the final models it produced got worse in a specific way that the training logs show clearly: they stopped trading. I cannot prove from this data alone that cloning was the only cause, and I will say exactly what the comparison can and cannot support. What I can show is that this implementation of self-imitation — cloning my own live trades, weighted by outcome, with a heavy pre-training pass — coincided with that collapse in the models that received the heavy dose, and not in the ones that did not. Live experience turned out to be valuable for a different reason, and I ended up rebuilding the loop around what the live data actually contains.


Author: Abdullah Uygar Tuna

New comment