Articles on data analysis and statistics in MQL5

icon

Articles on mathematical models and laws of probability are interesting for many traders. Mathematics is the basis of technical indicators, and statistics is required to analyze trading results and develop strategies.

Read about the fuzzy logic, digital filters, market profile, Kohonen maps, neural gas and many other tools that can be used for trading.

Add a new article
latest | best
preview
MQL5 Wizard Techniques you should know (Part 21): Testing with Economic Calendar Data

MQL5 Wizard Techniques you should know (Part 21): Testing with Economic Calendar Data

Economic Calendar Data is not available for testing with Expert Advisors within Strategy Tester, by default. We look at how Databases could help in providing a work around this limitation. So, for this article we explore how SQLite databases can be used to archive Economic Calendar news such that wizard assembled Expert Advisors can use this to generate trade signals.
preview
MQL5 Wizard Techniques you should know (Part 43): Reinforcement Learning with SARSA

MQL5 Wizard Techniques you should know (Part 43): Reinforcement Learning with SARSA

SARSA, which is an abbreviation for State-Action-Reward-State-Action is another algorithm that can be used when implementing reinforcement learning. So, as we saw with Q-Learning and DQN, we look into how this could be explored and implemented as an independent model rather than just a training mechanism, in wizard assembled Expert Advisors.
preview
Price Action Analysis Toolkit Development (Part 9): External Flow

Price Action Analysis Toolkit Development (Part 9): External Flow

This article explores a new dimension of analysis using external libraries specifically designed for advanced analytics. These libraries, like pandas, provide powerful tools for processing and interpreting complex data, enabling traders to gain more profound insights into market dynamics. By integrating such technologies, we can bridge the gap between raw data and actionable strategies. Join us as we lay the foundation for this innovative approach and unlock the potential of combining technology with trading expertise.
preview
Data Science and ML (Part 35): NumPy in MQL5 – The Art of Making Complex Algorithms with Less Code

Data Science and ML (Part 35): NumPy in MQL5 – The Art of Making Complex Algorithms with Less Code

NumPy library is powering almost all the machine learning algorithms to the core in Python programming language, In this article we are going to implement a similar module which has a collection of all the complex code to aid us in building sophisticated models and algorithms of any kind.
preview
Developing a Replay System — Market simulation (Part 16): New class system

Developing a Replay System — Market simulation (Part 16): New class system

We need to organize our work better. The code is growing, and if this is not done now, then it will become impossible. Let's divide and conquer. MQL5 allows the use of classes which will assist in implementing this task, but for this we need to have some knowledge about classes. Probably the thing that confuses beginners the most is inheritance. In this article, we will look at how to use these mechanisms in a practical and simple way.
preview
Creating a Trading Administrator Panel in MQL5 (Part IX): Code Organization (I)

Creating a Trading Administrator Panel in MQL5 (Part IX): Code Organization (I)

This discussion delves into the challenges encountered when working with large codebases. We will explore the best practices for code organization in MQL5 and implement a practical approach to enhance the readability and scalability of our Trading Administrator Panel source code. Additionally, we aim to develop reusable code components that can potentially benefit other developers in their algorithm development. Read on and join the conversation.
preview
Combinatorially Symmetric Cross Validation In MQL5

Combinatorially Symmetric Cross Validation In MQL5

In this article we present the implementation of Combinatorially Symmetric Cross Validation in pure MQL5, to measure the degree to which a overfitting may occure after optimizing a strategy using the slow complete algorithm of the Strategy Tester.
preview
Developing a Replay System — Market simulation (Part 07): First improvements (II)

Developing a Replay System — Market simulation (Part 07): First improvements (II)

In the previous article, we made some fixes and added tests to our replication system to ensure the best possible stability. We also started creating and using a configuration file for this system.
preview
Across Neighbourhood Search (ANS)

Across Neighbourhood Search (ANS)

The article reveals the potential of the ANS algorithm as an important step in the development of flexible and intelligent optimization methods that can take into account the specifics of the problem and the dynamics of the environment in the search space.
preview
Developing a Replay System (Part 47): Chart Trade Project (VI)

Developing a Replay System (Part 47): Chart Trade Project (VI)

Finally, our Chart Trade indicator starts interacting with the EA, allowing information to be transferred interactively. Therefore, in this article, we will improve the indicator, making it functional enough to be used together with any EA. This will allow us to access the Chart Trade indicator and work with it as if it were actually connected with an EA. But we will do it in a much more interesting way than before.
preview
MQL5 Wizard Techniques you should know (Part 07): Dendrograms

MQL5 Wizard Techniques you should know (Part 07): Dendrograms

Data classification for purposes of analysis and forecasting is a very diverse arena within machine learning and it features a large number of approaches and methods. This piece looks at one such approach, namely Agglomerative Hierarchical Classification.
preview
Population optimization algorithms: Resistance to getting stuck in local extrema (Part II)

Population optimization algorithms: Resistance to getting stuck in local extrema (Part II)

We continue our experiment that aims to examine the behavior of population optimization algorithms in the context of their ability to efficiently escape local minima when population diversity is low and reach global maxima. Research results are provided.
preview
Artificial Ecosystem-based Optimization (AEO) algorithm

Artificial Ecosystem-based Optimization (AEO) algorithm

The article considers a metaheuristic Artificial Ecosystem-based Optimization (AEO) algorithm, which simulates interactions between ecosystem components by creating an initial population of solutions and applying adaptive update strategies, and describes in detail the stages of AEO operation, including the consumption and decomposition phases, as well as different agent behavior strategies. The article introduces the features and advantages of this algorithm.
preview
The case for using a Composite Data Set this Q4 in weighing SPDR XLY's next performance

The case for using a Composite Data Set this Q4 in weighing SPDR XLY's next performance

We consider XLY, SPDR’s consumer discretionary spending ETF and see if with tools in MetaTrader’s IDE we can sift through an array of data sets in selecting what could work with a forecasting model with a forward outlook of not more than a year.
preview
Population optimization algorithms: Simulated Isotropic Annealing (SIA) algorithm. Part II

Population optimization algorithms: Simulated Isotropic Annealing (SIA) algorithm. Part II

The first part was devoted to the well-known and popular algorithm - simulated annealing. We have thoroughly considered its pros and cons. The second part of the article is devoted to the radical transformation of the algorithm, which turns it into a new optimization algorithm - Simulated Isotropic Annealing (SIA).
preview
Developing a multi-currency Expert Advisor (Part 7): Selecting a group based on forward period

Developing a multi-currency Expert Advisor (Part 7): Selecting a group based on forward period

Previously, we evaluated the selection of a group of trading strategy instances, with the aim of improving the results of their joint operation, only on the same time period, in which the optimization of individual instances was carried out. Let's see what happens in the forward period.
preview
From Novice to Expert: Animated News Headline Using MQL5 (I)

From Novice to Expert: Animated News Headline Using MQL5 (I)

News accessibility is a critical factor when trading on the MetaTrader 5 terminal. While numerous news APIs are available, many traders face challenges in accessing and integrating them effectively into their trading environment. In this discussion, we aim to develop a streamlined solution that brings news directly onto the chart—where it’s most needed. We'll accomplish this by building a News Headline Expert Advisor that monitors and displays real-time news updates from API sources.
preview
Royal Flush Optimization (RFO)

Royal Flush Optimization (RFO)

The original Royal Flush Optimization algorithm offers a new approach to solving optimization problems, replacing the classic binary coding of genetic algorithms with a sector-based approach inspired by poker principles. RFO demonstrates how simplifying basic principles can lead to an efficient and practical optimization method. The article presents a detailed analysis of the algorithm and test results.
preview
Category Theory in MQL5 (Part 21): Natural Transformations with LDA

Category Theory in MQL5 (Part 21): Natural Transformations with LDA

This article, the 21st in our series, continues with a look at Natural Transformations and how they can be implemented using linear discriminant analysis. We present applications of this in a signal class format, like in the previous article.
preview
Category Theory in MQL5 (Part 19): Naturality Square Induction

Category Theory in MQL5 (Part 19): Naturality Square Induction

We continue our look at natural transformations by considering naturality square induction. Slight restraints on multicurrency implementation for experts assembled with the MQL5 wizard mean we are showcasing our data classification abilities with a script. Principle applications considered are price change classification and thus its forecasting.
preview
Developing a Replay System (Part 72): An Unusual Communication (I)

Developing a Replay System (Part 72): An Unusual Communication (I)

What we create today will be difficult to understand. Therefore, in this article I will only talk about the initial stage. Please read this article carefully, it is an important prerequisite before we proceed to the next step. The purpose of this material is purely didactic as we will only study and master the presented concepts, without practical application.
preview
Matrix Factorization: The Basics

Matrix Factorization: The Basics

Since the goal here is didactic, we will proceed as simply as possible. That is, we will implement only what we need: matrix multiplication. You will see today that this is enough to simulate matrix-scalar multiplication. The most significant difficulty that many people encounter when implementing code using matrix factorization is this: unlike scalar factorization, where in almost all cases the order of the factors does not change the result, this is not the case when using matrices.
preview
MQL5 Wizard Techniques you should know (Part 34): Price-Embedding with an Unconventional RBM

MQL5 Wizard Techniques you should know (Part 34): Price-Embedding with an Unconventional RBM

Restricted Boltzmann Machines are a form of neural network that was developed in the mid 1980s at a time when compute resources were prohibitively expensive. At its onset, it relied on Gibbs Sampling and Contrastive Divergence in order to reduce dimensionality or capture the hidden probabilities/properties over input training data sets. We examine how Backpropagation can perform similarly when the RBM ‘embeds’ prices for a forecasting Multi-Layer-Perceptron.
preview
MQL5 Wizard Techniques you should know (Part 36): Q-Learning with Markov Chains

MQL5 Wizard Techniques you should know (Part 36): Q-Learning with Markov Chains

Reinforcement Learning is one of the three main tenets in machine learning, alongside supervised learning and unsupervised learning. It is therefore concerned with optimal control, or learning the best long-term policy that will best suit the objective function. It is with this back-drop, that we explore its possible role in informing the learning-process to an MLP of a wizard assembled Expert Advisor.
preview
The Group Method of Data Handling: Implementing the Combinatorial Algorithm in MQL5

The Group Method of Data Handling: Implementing the Combinatorial Algorithm in MQL5

In this article we continue our exploration of the Group Method of Data Handling family of algorithms, with the implementation of the Combinatorial Algorithm along with its refined incarnation, the Combinatorial Selective Algorithm in MQL5.
preview
Population optimization algorithms: Bird Swarm Algorithm (BSA)

Population optimization algorithms: Bird Swarm Algorithm (BSA)

The article explores the bird swarm-based algorithm (BSA) inspired by the collective flocking interactions of birds in nature. The different search strategies of individuals in BSA, including switching between flight, vigilance and foraging behavior, make this algorithm multifaceted. It uses the principles of bird flocking, communication, adaptability, leading and following to efficiently find optimal solutions.
preview
Developing a Replay System (Part 50): Things Get Complicated (II)

Developing a Replay System (Part 50): Things Get Complicated (II)

We will solve the chart ID problem and at the same time we will begin to provide the user with the ability to use a personal template for the analysis and simulation of the desired asset. The materials presented here are for didactic purposes only and should in no way be considered as an application for any purpose other than studying and mastering the concepts presented.
preview
Creating a Trading Administrator Panel in MQL5 (Part IX): Code Organization (III): Communication Module

Creating a Trading Administrator Panel in MQL5 (Part IX): Code Organization (III): Communication Module

Join us for an in-depth discussion on the latest advancements in MQL5 interface design as we unveil the redesigned Communications Panel and continue our series on building the New Admin Panel using modularization principles. We'll develop the CommunicationsDialog class step by step, thoroughly explaining how to inherit it from the Dialog class. Additionally, we'll leverage arrays and ListView class in our development. Gain actionable insights to elevate your MQL5 development skills—read through the article and join the discussion in the comments section!
preview
Data label for time series mining (Part 5):Apply and Test in EA Using Socket

Data label for time series mining (Part 5):Apply and Test in EA Using Socket

This series of articles introduces several time series labeling methods, which can create data that meets most artificial intelligence models, and targeted data labeling according to needs can make the trained artificial intelligence model more in line with the expected design, improve the accuracy of our model, and even help the model make a qualitative leap!
preview
Population optimization algorithms: Evolution Strategies, (μ,λ)-ES and (μ+λ)-ES

Population optimization algorithms: Evolution Strategies, (μ,λ)-ES and (μ+λ)-ES

The article considers a group of optimization algorithms known as Evolution Strategies (ES). They are among the very first population algorithms to use evolutionary principles for finding optimal solutions. We will implement changes to the conventional ES variants and revise the test function and test stand methodology for the algorithms.
preview
Developing a Replay System (Part 62): Playing the service (III)

Developing a Replay System (Part 62): Playing the service (III)

In this article, we will begin to address the issue of tick excess that can impact application performance when using real data. This excess often interferes with the correct timing required to construct a one-minute bar in the appropriate window.
preview
Artificial Bee Hive Algorithm (ABHA): Theory and methods

Artificial Bee Hive Algorithm (ABHA): Theory and methods

In this article, we will consider the Artificial Bee Hive Algorithm (ABHA) developed in 2009. The algorithm is aimed at solving continuous optimization problems. We will look at how ABHA draws inspiration from the behavior of a bee colony, where each bee has a unique role that helps them find resources more efficiently.
preview
Building a Trading System (Part 5): Managing Gains Through Structured Trade Exits

Building a Trading System (Part 5): Managing Gains Through Structured Trade Exits

For many traders, it's a familiar pain point: watching a trade come within a whisker of your profit target, only to reverse and hit your stop-loss. Or worse, seeing a trailing stop close you out at breakeven before the market surges toward your original target. This article focuses on using multiple entries at different Reward-to-Risk Ratios to systematically secure gains and reduce overall risk exposure.
preview
Data Science and ML (Part 34): Time series decomposition, Breaking the stock market down to the core

Data Science and ML (Part 34): Time series decomposition, Breaking the stock market down to the core

In a world overflowing with noisy and unpredictable data, identifying meaningful patterns can be challenging. In this article, we'll explore seasonal decomposition, a powerful analytical technique that helps separate data into its key components: trend, seasonal patterns, and noise. By breaking data down this way, we can uncover hidden insights and work with cleaner, more interpretable information.
preview
Reimagining Classic Strategies (Part 14): Multiple Strategy Analysis

Reimagining Classic Strategies (Part 14): Multiple Strategy Analysis

In this article, we continue our exploration of building an ensemble of trading strategies and using the MT5 genetic optimizer to tune the strategy parameters. Today, we analyzed the data in Python, showing our model could better predict which strategy would outperform, achieving higher accuracy than forecasting market returns directly. However, when we tested our application with its statistical models, our performance levels fell dismally. We subsequently discovered that the genetic optimizer unfortunately favored highly correlated strategies, prompting us to revise our method to keep vote weights fixed and focus optimization on indicator settings instead.
preview
Statistical Arbitrage Through Cointegrated Stocks (Part 4): Real-time Model Updating

Statistical Arbitrage Through Cointegrated Stocks (Part 4): Real-time Model Updating

This article describes a simple but comprehensive statistical arbitrage pipeline for trading a basket of cointegrated stocks. It includes a fully functional Python script for data download and storage; correlation, cointegration, and stationarity tests, along with a sample Metatrader 5 Service implementation for database updating, and the respective Expert Advisor. Some design choices are documented here for reference and for helping in the experiment replication.
preview
MQL5 Wizard Techniques you should know (Part 32): Regularization

MQL5 Wizard Techniques you should know (Part 32): Regularization

Regularization is a form of penalizing the loss function in proportion to the discrete weighting applied throughout the various layers of a neural network. We look at the significance, for some of the various regularization forms, this can have in test runs with a wizard assembled Expert Advisor.
preview
Developing a Replay System (Part 46): Chart Trade Project (V)

Developing a Replay System (Part 46): Chart Trade Project (V)

Tired of wasting time searching for that very file that you application needs in order to work? How about including everything in the executable? This way you won't have to search for the things. I know that many people use this form of distribution and storage, but there is a much more suitable way. At least as far as the distribution of executable files and their storage is concerned. The method that will be presented here can be very useful, since you can use MetaTrader 5 itself as an excellent assistant, as well as MQL5. Furthermore, it is not that difficult to understand.
preview
Developing a Replay System (Part 29): Expert Advisor project — C_Mouse class (III)

Developing a Replay System (Part 29): Expert Advisor project — C_Mouse class (III)

After improving the C_Mouse class, we can focus on creating a class designed to create a completely new framework fr our analysis. We will not use inheritance or polymorphism to create this new class. Instead, we will change, or better said, add new objects to the price line. That's what we will do in this article. In the next one, we will look at how to change the analysis. All this will be done without changing the code of the C_Mouse class. Well, actually, it would be easier to achieve this using inheritance or polymorphism. However, there are other methods to achieve the same result.
preview
Developing a Replay System (Part 43): Chart Trade Project (II)

Developing a Replay System (Part 43): Chart Trade Project (II)

Most people who want or dream of learning to program don't actually have a clue what they're doing. Their activity consists of trying to create things in a certain way. However, programming is not about tailoring suitable solutions. Doing it this way can create more problems than solutions. Here we will be doing something more advanced and therefore different.