Analysis of test results and optimisation in the MetaTrader 5 strategy tester - page 12

 

On a server with 12 threads and 128GB of RAM, when optimising in "Every tick based on real ticks" mode, from 2019 to now, runs started crashing due to lack of RAM. Does the tester really load on each core the entire real tick history into memory in one chunk? While it is guaranteed to be the same for all threads, and only local agents are used, network and cloud are not used. I.e. it is necessary and sufficient to load these quotes into memory once and then access them from all threads in read mode. But no, it doesn't seem to work that way.

I switched off 7 threads out of 12, everything went fine

 
Ilya Malev #:

Does the tester really load on each core the entire history of real ticks into memory in one chunk?

I recently described working with memory in optimisation mode. There may be several variants there.

While it is guaranteed to be the same for all threads, and only local agents are used, network and cloud are not used. I.e. it is necessary and sufficient to load these quotes into memory once and then access them from all threads in read mode. But no, it doesn't seem to work that way.

In particular, EAToMath is made for this purpose. There, at any history size (only for one symbol) and number of cores, memory will always be enough for everything.

Работа MT5-тестера с RAM/Disk.
Работа MT5-тестера с RAM/Disk.
  • 2025.09.01
  • www.mql5.com
Многоядерный MT5-тестер имеет свои особенности работы с памятью и диском, знание которых дает понимание его ограничений на запускаемых конфигурациях и объясняет некоторые возникающие ошибки во время
 
fxsaber #:

I recently described working with memory in optimisation mode. There may be several variants there.

In particular, EAToMath is made for this purpose. There, at any history size (for one symbol only) and number of cores, there will always be enough memory for everything.

Thank you of course, this is cool work, but I am not ready to move to mat calculations yet. I would like the developers, who implement a lot of things that are supposedly necessary, to pay attention also to what is necessary and can be improved to the benefit of real traders.
 
fxsaber #:

In particular, EAToMath is made for this purpose. There, for any history size (for one symbol only) and number of cores, there will always be enough memory for everything.

Only one question remains - in the description of EAToMath it seems to say that "Trading Indicator Expert Advisors" are supported, but there is nothing in the code or in the example for the work of indicators. Where am I wrong?

 
Ilya Malev #:

On a server with 12 threads and 128GB of RAM, when optimising in "Every tick based on real ticks" mode, from 2019 to now, runs started crashing due to lack of RAM. Does the tester really load on each core the entire real tick history into memory in one chunk? While it is guaranteed to be the same for all threads, and only local agents are used, network and cloud are not used. I.e. it is necessary and sufficient to load these quotes into memory once and then access them from all threads in read mode. But no, it doesn't seem to work that way.

I switched off 7 threads out of 12, everything went fine

Yes, there have been such suggestions more than once, but the prioritisation by task in MQ has always been weird.

To be fair, agents are separate processes, so access to the common tick database (if they ever manage to make it) should be provided from them, not from threads of one process.

Новая версия платформы MetaTrader 5 build 5120: улучшения и исправления - Для тестирования по большим данным нужно использовать ArrayInitialize что ли?
Новая версия платформы MetaTrader 5 build 5120: улучшения и исправления - Для тестирования по большим данным нужно использовать ArrayInitialize что ли?
  • 2025.06.19
  • www.mql5.com
что ради ускорения тестирования тиковая и баровая история загружается в оперативку каждому агенту. всем агентам или по 1 копии для 8 агентов или по 1й копии на 1 канал в оперативке - тогда замедление тоже будет минимальным
 
Stanislav Korotky #:

Only one question remains - it seems to be written in the description of EAToMath that "Trading Indicator Expert Advisors" are supported, but there is nothing in the code and example for indicators to work. Where am I wrong?

If you do not have the matrix mode, they are supported. In the matrix mode, they are not. In the first case, only the regular Tester is responsible for using RAM, in the second case - EAToMath.


It is a mystery to me why SymbolInfoTick is not enough for trading. If you need bars, you only need to write your own variant of CopyBuffer from SymbolInfoTick. Write your own indicators through this pad. Then SymbolInfoTick will always be enough for indicators in the Tester, including EAToMath. And in the Terminal they will work normally - through the terminal CopyBuffer.

 
fxsaber #:

It is a mystery to me why SymbolInfoTick is not enough for trading. If you need bars, you only need to write your own variant of CopyBuffer from SymbolInfoTick. Write your own indicators through this pad. Then SymbolInfoTick will always be enough for indicators in the Tester, including EAToMath. And in the Terminal they will work normally - through the terminal CopyBuffer.

Because such a pad is a lot of routine programming and a long struggle with "brakes", which will definitely occur in this homemade implementation. No one has come to this yet, although I saw a similar product in the market long ago, but then I couldn't find it.
 
Stanislav Korotky #:
Because such a pad is a bunch of routine programming and a long struggle with "brakes", which in this homemade implementation will definitely occur.
Homemade testers are not in demand. It is a useless job to build additional functionality for them. I really have enough of what I have laid out.
 
fxsaber #:

It's a mystery to me why SymbolInfoTick is missing for trading.

Advertising.

 

How to understand these messages without a reason, where to find out the specific reason?