Discussing the article: "Measuring the Probability of Backtest Overfitting of Your MetaTrader 5 Optimization (Part 1): The CSCV Engine in MQL5"

New comment

Check out the new article: Measuring the Probability of Backtest Overfitting of Your MetaTrader 5 Optimization (Part 1): The CSCV Engine in MQL5.

This article implements the Probability of Backtest Overfitting in MQL5 with Combinatorially Symmetric Cross-Validation. It shows how to build the bar-by-bar returns matrix of a parameter grid, aggregate it by blocks and rank the in-sample winner out-of-sample across every split of the history. You get a single probability that this winner ends up in the lower half of the grid out-of-sample, along with the MQL5 code to compute it on your own optimizations.

In this article, we measure the Probability of Backtest Overfitting of your MetaTrader 5 optimization. In plain terms, it tells you how likely your optimization process is to have selected noise rather than a real signal.

The measure is the Probability of Backtest Overfitting (PBO) of Bailey, Borwein, López de Prado and Zhu (2015), computed through their Combinatorially Symmetric Cross-Validation (CSCV) procedure. You get one number, and above 5% you reject the optimization procedure and do not trade the strategy it selected. That 5% is the customary cut-off the authors suggest, by analogy with the usual 5% significance level, so it is a convention rather than something the method derives.

This is worth doing every time you optimize. If you run sweeps in the Strategy Tester, you can run the code below on the returns behind them and estimate how likely it is that the retained parameters are due to luck rather than to a structural market effect. Optimizing is not the problem, and optimizing without knowing which of the two you found is.

The method is not mine. My master’s thesis, in quantitative asset and risk management, applied this same framework to a panel of technical indicators on the S&P 500 and the Nasdaq 100, and it is published on SSRN. What is new here is the MetaTrader 5 implementation.

Author: Astralys LLC

Hi everyone, I'm the author of this article. If you have any questions about it, whatever they are, don't hesitate to ask here.
New comment