Articles on data analysis and statistics in MQL5

In the previous article, we made some fixes and added tests to our replication system to ensure the best possible stability. We also started creating and using a configuration file for this system.

The first part was devoted to the well-known and popular algorithm - simulated annealing. We have thoroughly considered its pros and cons. The second part of the article is devoted to the radical transformation of the algorithm, which turns it into a new optimization algorithm - Simulated Isotropic Annealing (SIA).

Do you want to know how to benefit from the difference in interest rates? This article considers how to use swap arbitrage in Forex to earn stable profit every night, creating a portfolio that is resistant to market fluctuations.

In this article we present an algorithm for permuting price bars and detail how permutation tests can be used to recognize instances where strategy performance has been fabricated to deceive potential buyers of Expert Advisors.

In this article, we will get acquainted with the ALGLIB library optimization methods for MQL5. The article includes simple and clear examples of using ALGLIB to solve optimization problems, which will make mastering the methods as accessible as possible. We will take a detailed look at the connection of such algorithms as BLEIC, L-BFGS and NS, and use them to solve a simple test problem.

We continue our experiment that aims to examine the behavior of population optimization algorithms in the context of their ability to efficiently escape local minima when population diversity is low and reach global maxima. Research results are provided.

This is an improved chaotic optimization algorithm (COA) that combines the effects of chaos with adaptive search mechanisms. The algorithm uses a set of chaotic maps and inertial components to explore the search space. The article reveals the theoretical foundations of chaotic methods of financial optimization.

What we create today will be difficult to understand. Therefore, in this article I will only talk about the initial stage. Please read this article carefully, it is an important prerequisite before we proceed to the next step. The purpose of this material is purely didactic as we will only study and master the presented concepts, without practical application.

Here we are going to develop a script from scratch that simplifies unloading print screens of deals for analyzing trading entries. All the necessary information on a single deal is to be conveniently displayed on one chart with the ability to draw different timeframes.

Finally, our Chart Trade indicator starts interacting with the EA, allowing information to be transferred interactively. Therefore, in this article, we will improve the indicator, making it functional enough to be used together with any EA. This will allow us to access the Chart Trade indicator and work with it as if it were actually connected with an EA. But we will do it in a much more interesting way than before.

This article presents a sample MQL5 Service implementation for updating a newly created database used as source for data analysis and for trading a basket of cointegrated stocks. The rationale behind the database design is explained in detail and the data dictionary is documented for reference. MQL5 and Python scripts are provided for the database creation, schema initialization, and market data insertion.

This article proposes using Rolling Windows Eigenvector Comparison for early imbalance diagnostics and portfolio rebalancing in a mean-reversion statistical arbitrage strategy based on cointegrated stocks. It contrasts this technique with traditional In-Sample/Out-of-Sample ADF validation, showing that eigenvector shifts can signal the need for rebalancing even when IS/OOS ADF still indicates a stationary spread. While the method is intended mainly for live trading monitoring, the article concludes that eigenvector comparison could also be integrated into the scoring system—though its actual contribution to performance remains to be tested.

In this article, we will explore and implement, methods for assessing model quality that utilize a single dataset as both training and validation sets.

Restricted Boltzmann Machines are a form of neural network that was developed in the mid 1980s at a time when compute resources were prohibitively expensive. At its onset, it relied on Gibbs Sampling and Contrastive Divergence in order to reduce dimensionality or capture the hidden probabilities/properties over input training data sets. We examine how Backpropagation can perform similarly when the RBM ‘embeds’ prices for a forecasting Multi-Layer-Perceptron.

We consider XLY, SPDR’s consumer discretionary spending ETF and see if with tools in MetaTrader’s IDE we can sift through an array of data sets in selecting what could work with a forecasting model with a forward outlook of not more than a year.

We will solve the chart ID problem and at the same time we will begin to provide the user with the ability to use a personal template for the analysis and simulation of the desired asset. The materials presented here are for didactic purposes only and should in no way be considered as an application for any purpose other than studying and mastering the concepts presented.

In this article, we will begin to address the issue of tick excess that can impact application performance when using real data. This excess often interferes with the correct timing required to construct a one-minute bar in the appropriate window.

The article examines the theoretical and practical aspects of the singular spectrum analysis (SSA) method, which is an efficient method of time series analysis that allows one to represent the complex structure of a series as a decomposition into simple components, such as trend, seasonal (periodic) fluctuations and noise.

The article considers Boids algorithm based on unique examples of animal flocking behavior. In turn, the Boids algorithm serves as the basis for the creation of the whole class of algorithms united under the name "Swarm Intelligence".

In this article, we will consider the Artificial Bee Hive Algorithm (ABHA) developed in 2009. The algorithm is aimed at solving continuous optimization problems. We will look at how ABHA draws inspiration from the behavior of a bee colony, where each bee has a unique role that helps them find resources more efficiently.

This article presents the development of an arbitrage analysis panel in MQL5. How to get fair exchange rates on Forex in different ways? Create an indicator to obtain deviations of market prices from fair exchange rates, as well as to assess the benefits of arbitrage ways of exchanging one currency for another (as in triangular arbitrage).

Financial markets are not perfectly balanced. Some markets are bullish, some are bearish, and some exhibit some ranging behaviors indicating uncertainty in either direction, this unbalanced information when used to train machine learning models can be misleading as the markets change frequently. In this article, we are going to discuss several ways to tackle this issue.

This article explores the powerful role of matrix factorization in algorithmic trading, specifically within MQL5 applications. From regression models to multi-target classifiers, we walk through practical examples that demonstrate how easily these techniques can be integrated using built-in MQL5 functions. Whether you're predicting price direction or modeling indicator behavior, this guide lays a strong foundation for building intelligent trading systems using matrix methods.

In this article, we develop a custom indicator in MQL5 for hybrid Time Price Opportunity (TPO) market profiles, supporting multiple session timeframes such as intraday, daily, weekly, monthly, and fixed periods with timezone adjustments. The indicator quantizes prices into a grid, tracks session data including highs, lows, opens, and closes, and calculates key elements like the point of control and value area based on TPO counts. It renders profiles visually on the chart with customizable colors for TPO letters, single prints, value areas, POC, and close markers, enabling detailed session analysis

Continuing tries to decipher price movements... What about linguistic analysis of the "market dictionary" that we get by converting the binary price code to BIP39? In this article, we will delve into an innovative approach to exchange data analysis and consider how modern natural language processing techniques can be applied to the market language.

Replay buffers in Reinforcement Learning are particularly important with off-policy algorithms like DQN or SAC. This then puts the spotlight on the sampling process of this memory-buffer. While default options with SAC, for instance, use random selection from this buffer, Prioritized Experience Replay buffers fine tune this by sampling from the buffer based on a TD-score. We review the importance of Reinforcement Learning, and, as always, examine just this hypothesis (not the cross-validation) in a wizard assembled Expert Advisor.

Regularization is a form of penalizing the loss function in proportion to the discrete weighting applied throughout the various layers of a neural network. We look at the significance, for some of the various regularization forms, this can have in test runs with a wizard assembled Expert Advisor.

We are close to completing this challenge. However, before we begin, I want you to try to understand these two articles—this one and the previous one. That way, you will truly understand the next article, in which I will cover exclusively the part related to MQL5 programming. But I will also try to make it understandable. If you do not understand these last two articles, it will be difficult for you to understand the next one, because the material accumulates. The more things there are to do, the more you need to create and understand in order to achieve the goal.

In this article, we discuss how statistical causality can be applied to identify predictive variables. We will explore the link between causality and transfer entropy, as well as present MQL5 code for detecting directional transfers of information between two variables.

How can a trader manage capital? How can a trader and investor keep track of expenses, income, assets, and liabilities? I am not just going to introduce you to accounting software; I am going to show you a tool that might become your reliable financial navigator in the stormy sea of trading.