Articles on data analysis and statistics in MQL5

icon

Articles on mathematical models and laws of probability are interesting for many traders. Mathematics is the basis of technical indicators, and statistics is required to analyze trading results and develop strategies.

Read about the fuzzy logic, digital filters, market profile, Kohonen maps, neural gas and many other tools that can be used for trading.

Add a new article
latest | best
preview

Permuting price bars in MQL5

In this article we present an algorithm for permuting price bars and detail how permutation tests can be used to recognize instances where strategy performance has been fabricated to deceive potential buyers of Expert Advisors.
preview

Pipelines in MQL5

In this piece, we look at a key data preparation step for machine learning that is gaining rapid significance. Data Preprocessing Pipelines. These in essence are a streamlined sequence of data transformation steps that prepare raw data before it is fed to a model. As uninteresting as this may initially seem to the uninducted, this ‘data standardization’ not only saves on training time and execution costs, but it goes a long way in ensuring better generalization. In this article we are focusing on some SCIKIT-LEARN preprocessing functions, and while we are not exploiting the MQL5 Wizard, we will return to it in coming articles.
preview

Visualizing deals on a chart (Part 1): Selecting a period for analysis

Here we are going to develop a script from scratch that simplifies unloading print screens of deals for analyzing trading entries. All the necessary information on a single deal is to be conveniently displayed on one chart with the ability to draw different timeframes.
preview

Data Science and ML (Part 46): Stock Markets Forecasting Using N-BEATS in Python

N-BEATS is a revolutionary deep learning model designed for time series forecasting. It was released to surpass classical models for time series forecasting such as ARIMA, PROPHET, VAR, etc. In this article, we are going to discuss this model and use it in predicting the stock market.
preview

Most notable Artificial Cooperative Search algorithm modifications (ACSm)

Here we will consider the evolution of the ACS algorithm: three modifications aimed at improving the convergence characteristics and the algorithm efficiency. Transformation of one of the leading optimization algorithms. From matrix modifications to revolutionary approaches regarding population formation.
preview

Quantum computing and trading: A fresh approach to price forecasts

The article describes an innovative approach to forecasting price movements in financial markets using quantum computing. The main focus is on the application of the Quantum Phase Estimation (QPE) algorithm to find prototypes of price patterns allowing traders to significantly speed up the market data analysis.
preview

Developing a Replay System (Part 33): Order System (II)

Today we will continue to develop the order system. As you will see, we will be massively reusing what has already been shown in other articles. Nevertheless, you will receive a small reward in this article. First, we will develop a system that can be used with a real trading server, both from a demo account or from a real one. We will make extensive use of the MetaTrader 5 platform, which will provide us with all the necessary support from the beginning.
preview

Resampling techniques for prediction and classification assessment in MQL5

In this article, we will explore and implement, methods for assessing model quality that utilize a single dataset as both training and validation sets.
preview

Statistical Arbitrage Through Cointegrated Stocks (Part 2): Expert Advisor, Backtests, and Optimization

This article presents a sample Expert Advisor implementation for trading a basket of four Nasdaq stocks. The stocks were initially filtered based on Pearson correlation tests. The filtered group was then tested for cointegration with Johansen tests. Finally, the cointegrated spread was tested for stationarity with the ADF and KPSS tests. Here we will see some notes about this process and the results of the backtests after a small optimization.
preview

MQL5 Wizard Techniques you should know (Part 43): Reinforcement Learning with SARSA

SARSA, which is an abbreviation for State-Action-Reward-State-Action is another algorithm that can be used when implementing reinforcement learning. So, as we saw with Q-Learning and DQN, we look into how this could be explored and implemented as an independent model rather than just a training mechanism, in wizard assembled Expert Advisors.
preview

Market Simulation (Part 01): Cross Orders (I)

Today we will begin the second stage, where we will look at the market replay/simulation system. First, we will show a possible solution for cross orders. I will show you the solution, but it is not final yet. It will be a possible solution to a problem that we will need to solve in the near future.
preview

Across Neighbourhood Search (ANS)

The article reveals the potential of the ANS algorithm as an important step in the development of flexible and intelligent optimization methods that can take into account the specifics of the problem and the dynamics of the environment in the search space.
preview

Big Bang - Big Crunch (BBBC) algorithm

The article presents the Big Bang - Big Crunch method, which has two key phases: cyclic generation of random points and their compression to the optimal solution. This approach combines exploration and refinement, allowing us to gradually find better solutions and open up new optimization opportunities.
preview

Developing a Replay System (Part 69): Getting the Time Right (II)

Today we will look at why we need the iSpread feature. At the same time, we will understand how the system informs us about the remaining time of the bar when there is not a single tick available for it. The content presented here is intended solely for educational purposes. Under no circumstances should the application be viewed for any purpose other than to learn and master the concepts presented.
preview
Reimagining Classic Strategies (Part 14): Multiple Strategy Analysis

Reimagining Classic Strategies (Part 14): Multiple Strategy Analysis

In this article, we continue our exploration of building an ensemble of trading strategies and using the MT5 genetic optimizer to tune the strategy parameters. Today, we analyzed the data in Python, showing our model could better predict which strategy would outperform, achieving higher accuracy than forecasting market returns directly. However, when we tested our application with its statistical models, our performance levels fell dismally. We subsequently discovered that the genetic optimizer unfortunately favored highly correlated strategies, prompting us to revise our method to keep vote weights fixed and focus optimization on indicator settings instead.
preview
From Novice to Expert: Animated News Headline Using MQL5 (I)

From Novice to Expert: Animated News Headline Using MQL5 (I)

News accessibility is a critical factor when trading on the MetaTrader 5 terminal. While numerous news APIs are available, many traders face challenges in accessing and integrating them effectively into their trading environment. In this discussion, we aim to develop a streamlined solution that brings news directly onto the chart—where it’s most needed. We'll accomplish this by building a News Headline Expert Advisor that monitors and displays real-time news updates from API sources.
preview
Population optimization algorithms: Evolution Strategies, (μ,λ)-ES and (μ+λ)-ES

Population optimization algorithms: Evolution Strategies, (μ,λ)-ES and (μ+λ)-ES

The article considers a group of optimization algorithms known as Evolution Strategies (ES). They are among the very first population algorithms to use evolutionary principles for finding optimal solutions. We will implement changes to the conventional ES variants and revise the test function and test stand methodology for the algorithms.
preview
Data Science and ML (Part 34): Time series decomposition, Breaking the stock market down to the core

Data Science and ML (Part 34): Time series decomposition, Breaking the stock market down to the core

In a world overflowing with noisy and unpredictable data, identifying meaningful patterns can be challenging. In this article, we'll explore seasonal decomposition, a powerful analytical technique that helps separate data into its key components: trend, seasonal patterns, and noise. By breaking data down this way, we can uncover hidden insights and work with cleaner, more interpretable information.
preview
Category Theory in MQL5 (Part 12): Orders

Category Theory in MQL5 (Part 12): Orders

This article which is part of a series that follows Category Theory implementation of Graphs in MQL5, delves in Orders. We examine how concepts of Order-Theory can support monoid sets in informing trade decisions by considering two major ordering types.
preview
Integrating MQL5 with data processing packages (Part 1): Advanced Data analysis and Statistical Processing

Integrating MQL5 with data processing packages (Part 1): Advanced Data analysis and Statistical Processing

Integration enables seamless workflow where raw financial data from MQL5 can be imported into data processing packages like Jupyter Lab for advanced analysis including statistical testing.
preview
Anarchic Society Optimization (ASO) algorithm

Anarchic Society Optimization (ASO) algorithm

In this article, we will get acquainted with the Anarchic Society Optimization (ASO) algorithm and discuss how an algorithm based on the irrational and adventurous behavior of participants in an anarchic society (an anomalous system of social interaction free from centralized power and various kinds of hierarchies) is able to explore the solution space and avoid the traps of local optimum. The article presents a unified ASO structure applicable to both continuous and discrete problems.
preview
Chemical reaction optimization (CRO) algorithm (Part I): Process chemistry in optimization

Chemical reaction optimization (CRO) algorithm (Part I): Process chemistry in optimization

In the first part of this article, we will dive into the world of chemical reactions and discover a new approach to optimization! Chemical reaction optimization (CRO) uses principles derived from the laws of thermodynamics to achieve efficient results. We will reveal the secrets of decomposition, synthesis and other chemical processes that became the basis of this innovative method.
preview
Animal Migration Optimization (AMO) algorithm

Animal Migration Optimization (AMO) algorithm

The article is devoted to the AMO algorithm, which models the seasonal migration of animals in search of optimal conditions for life and reproduction. The main features of AMO include the use of topological neighborhood and a probabilistic update mechanism, which makes it easy to implement and flexible for various optimization tasks.
preview
Self Optimizing Expert Advisors in MQL5 (Part 12): Building Linear Classifiers Using Matrix Factorization

Self Optimizing Expert Advisors in MQL5 (Part 12): Building Linear Classifiers Using Matrix Factorization

This article explores the powerful role of matrix factorization in algorithmic trading, specifically within MQL5 applications. From regression models to multi-target classifiers, we walk through practical examples that demonstrate how easily these techniques can be integrated using built-in MQL5 functions. Whether you're predicting price direction or modeling indicator behavior, this guide lays a strong foundation for building intelligent trading systems using matrix methods.
preview
Hybridization of population algorithms. Sequential and parallel structures

Hybridization of population algorithms. Sequential and parallel structures

Here we will dive into the world of hybridization of optimization algorithms by looking at three key types: strategy mixing, sequential and parallel hybridization. We will conduct a series of experiments combining and testing relevant optimization algorithms.
preview
Population optimization algorithms: Boids Algorithm

Population optimization algorithms: Boids Algorithm

The article considers Boids algorithm based on unique examples of animal flocking behavior. In turn, the Boids algorithm serves as the basis for the creation of the whole class of algorithms united under the name "Swarm Intelligence".
preview
MQL5 Wizard Techniques you should know (Part 55): SAC with Prioritized Experience Replay

MQL5 Wizard Techniques you should know (Part 55): SAC with Prioritized Experience Replay

Replay buffers in Reinforcement Learning are particularly important with off-policy algorithms like DQN or SAC. This then puts the spotlight on the sampling process of this memory-buffer. While default options with SAC, for instance, use random selection from this buffer, Prioritized Experience Replay buffers fine tune this by sampling from the buffer based on a TD-score. We review the importance of Reinforcement Learning, and, as always, examine just this hypothesis (not the cross-validation) in a wizard assembled Expert Advisor.
preview
Spurious Regressions in Python

Spurious Regressions in Python

Spurious regressions occur when two time series exhibit a high degree of correlation purely by chance, leading to misleading results in regression analysis. In such cases, even though variables may appear to be related, the correlation is coincidental and the model may be unreliable.
preview
Population optimization algorithms: Spiral Dynamics Optimization (SDO) algorithm

Population optimization algorithms: Spiral Dynamics Optimization (SDO) algorithm

The article presents an optimization algorithm based on the patterns of constructing spiral trajectories in nature, such as mollusk shells - the spiral dynamics optimization (SDO) algorithm. I have thoroughly revised and modified the algorithm proposed by the authors. The article will consider the necessity of these changes.
preview
Ordinal Encoding for Nominal Variables

Ordinal Encoding for Nominal Variables

In this article, we discuss and demonstrate how to convert nominal predictors into numerical formats that are suitable for machine learning algorithms, using both Python and MQL5.
preview
Economic forecasts: Exploring the Python potential

Economic forecasts: Exploring the Python potential

How to use World Bank economic data for forecasts? What happens when you combine AI models and economics?
preview
Ensemble methods to enhance numerical predictions in MQL5

Ensemble methods to enhance numerical predictions in MQL5

In this article, we present the implementation of several ensemble learning methods in MQL5 and examine their effectiveness across different scenarios.
preview
From Novice to Expert: Mastering Detailed Trading Reports with Reporting EA

From Novice to Expert: Mastering Detailed Trading Reports with Reporting EA

In this article, we delve into enhancing the details of trading reports and delivering the final document via email in PDF format. This marks a progression from our previous work, as we continue exploring how to harness the power of MQL5 and Python to generate and schedule trading reports in the most convenient and professional formats. Join us in this discussion to learn more about optimizing trading report generation within the MQL5 ecosystem.
preview
Forecasting exchange rates using classic machine learning methods: Logit and Probit models

Forecasting exchange rates using classic machine learning methods: Logit and Probit models

In the article, an attempt is made to build a trading EA for predicting exchange rate quotes. The algorithm is based on classical classification models - logistic and probit regression. The likelihood ratio criterion is used as a filter for trading signals.
preview
Functions for activating neurons during training: The key to fast convergence?

Functions for activating neurons during training: The key to fast convergence?

This article presents a study of the interaction of different activation functions with optimization algorithms in the context of neural network training. Particular attention is paid to the comparison of the classical ADAM and its population version when working with a wide range of activation functions, including the oscillating ACON and Snake functions. Using a minimalistic MLP (1-1-1) architecture and a single training example, the influence of activation functions on the optimization is isolated from other factors. The article proposes an approach to manage network weights through the boundaries of activation functions and a weight reflection mechanism, which allows avoiding problems with saturation and stagnation in training.
preview
MQL5 Wizard Techniques you should know (14): Multi Objective Timeseries Forecasting with STF

MQL5 Wizard Techniques you should know (14): Multi Objective Timeseries Forecasting with STF

Spatial Temporal Fusion which is using both ‘space’ and time metrics in modelling data is primarily useful in remote-sensing, and a host of other visual based activities in gaining a better understanding of our surroundings. Thanks to a published paper, we take a novel approach in using it by examining its potential to traders.
preview
Applying Localized Feature Selection in Python and MQL5

Applying Localized Feature Selection in Python and MQL5

This article explores a feature selection algorithm introduced in the paper 'Local Feature Selection for Data Classification' by Narges Armanfard et al. The algorithm is implemented in Python to build binary classifier models that can be integrated with MetaTrader 5 applications for inference.
preview
Brain Storm Optimization algorithm (Part I): Clustering

Brain Storm Optimization algorithm (Part I): Clustering

In this article, we will look at an innovative optimization method called BSO (Brain Storm Optimization) inspired by a natural phenomenon called "brainstorming". We will also discuss a new approach to solving multimodal optimization problems the BSO method applies. It allows finding multiple optimal solutions without the need to pre-determine the number of subpopulations. We will also consider the K-Means and K-Means++ clustering methods.
preview
Developing a Replay System (Part 52): Things Get Complicated (IV)

Developing a Replay System (Part 52): Things Get Complicated (IV)

In this article, we will change the mouse pointer to enable the interaction with the control indicator to ensure reliable and stable operation.
preview
ALGLIB library optimization methods (Part I)

ALGLIB library optimization methods (Part I)

In this article, we will get acquainted with the ALGLIB library optimization methods for MQL5. The article includes simple and clear examples of using ALGLIB to solve optimization problems, which will make mastering the methods as accessible as possible. We will take a detailed look at the connection of such algorithms as BLEIC, L-BFGS and NS, and use them to solve a simple test problem.