Bug in strategy tester

 

Hi there.

I searched the forums but could not find an answer or similar existing topic.

The issue I have for a while with MT5 is freezing after selecting "Run single test" from optimization results window. 

The version I have is latest version from my broker (version 5.00 build 4153 dated 22 Jan 2024), but the same issue has been around for a while. Every new version I hoped it would have been fixed but to no avail. My system is Windows 10, 8 GB RAM. 

Issue happens whether I test my code, or example code such as "Expert MAMA" shipped with MT5.

Steps of the issue as follows:

  1. Run strategy tester with genetic optimization and forward testing, using local farm
  2. Optimizer completes
  3. Select on case from the list and right click "Run single test"
  4. Tester starts, then MT5 becomes non-responsive.
  5. I have to terminate MT5 from task manager
  6. This happens most of the time. After restart, try again from the list of cases, and sometimes it works, sometimes it freezes again. 

Previous versions of MT5 used to restart quicker, and when I tried to run single case from the list, I could run it usually.

But the latest version of MT5 restarts much slower, and also when I try to run the case again it often freezes. 

This is seriously affecting my development. Can anybody help with this please? Any workarounds are also fine as long as it resolved this. Thank you in advance.


PS: I checked the log file in the tester folder, and it seems like the case has run and finished. Head and tail excerpts from log file belonging to the example single run below:

CS 0 19:52:42.828 127.0.0.1 login (build 4153)

CS 0 19:52:42.841 Tester account info found with currency USD

CS 0 19:52:42.841 Network 1478 bytes of tester parameters loaded

CS 0 19:52:42.841 Network 4804 bytes of input parameters loaded

CS 0 19:52:42.847 Tester 1297 Mb available, 16 blocks set for ticks generating

CS 0 19:52:42.847 Tester initial deposit 500.00 USD, leverage 1:25

CS 0 19:52:42.850 Tester successfully initialized

CS 0 19:52:42.850 Network 930 bytes of total initialization data received

CS 0 19:52:42.850 Tester Intel Core i5-3337U  @ 1.80GHz, 8073 MB

CS 0 19:52:42.850 Symbols Gasoline: symbol to be synchronized

CS 0 19:52:42.851 Symbols Gasoline: symbol synchronized already, 22 bytes received

CS 0 19:52:42.858 History Gasoline: load 31 bytes of history data to synchronize in 0:00:00.003

CS 0 19:52:42.858 History Gasoline: history synchronized from 2021.06.03 to 2024.03.07

CS 3 19:52:42.862 Tester Gasoline: start time changed to 2021.06.11 00:00 to provide data at beginning

CS 0 19:52:42.862 History Gasoline,H1: history cache allocated for 10628 bars and contains 119 bars from 2021.06.03 15:00 to 2021.06.10 23:00

CS 0 19:52:42.862 History Gasoline,H1: history begins from 2021.06.03 15:00

CS 0 19:52:42.862 Tester Gasoline,H1 (____-Live): OHLC bar states generating. OnTick executed on the bar begin only

CS 0 19:52:42.862 Tester execution delay discarded because of open prices testing mode

CS 0 19:52:42.862 Tester Gasoline,H1: testing of Experts\Advisors\ExpertMAMA.ex5 from 2021.01.01 00:00 to 2023.02.14 00:00 started with inputs:

......


CS 0 19:52:44.128 Tester final balance 518.84 USD

CS 0 19:52:44.135 Tester Gasoline,H1: 24404 ticks, 6284 bars generated. Environment synchronized in 0:00:00.841. Test passed in 0:00:00.465 (including ticks preprocessing 0:00:00.187).

CS 0 19:52:44.135 Tester Gasoline,H1: total time from login to stop testing 0:00:01.306 (including 0:00:00.841 for history data synchronization)

CS 0 19:52:44.135 Tester 437 Mb memory used including 0.94 Mb of history data, 64 Mb of tick data

CS 0 19:52:44.135 Tester log file "C:\Users\___\Agent-127.0.0.1-3000\logs\20240308.log" written

CS 0 19:52:44.140 Tester test Experts\Advisors\ExpertMAMA.ex5 on Gasoline,H1 thread finished

CS 0 19:54:51.789 127.0.0.1 prepare for shutdown

CS 0 19:54:51.790 127.0.0.1 shutdown finished

CS 0 20:00:19.629 Server MetaTester 5 stopped



The Fundamentals of Testing in MetaTrader 5
The Fundamentals of Testing in MetaTrader 5
  • www.mql5.com
What are the differences between the three modes of testing in MetaTrader 5, and what should be particularly looked for? How does the testing of an EA, trading simultaneously on multiple instruments, take place? When and how are the indicator values calculated during testing, and how are the events handled? How to synchronize the bars from different instruments during testing in an "open prices only" mode? This article aims to provide answers to these and many other questions.
 
I have exactly the same problem. Couldn't find a fix yet.
 
Emanuel Cavalcante Amorim Filho #:
I have exactly the same problem. Couldn't find a fix yet.

It's good to hear that as it confirms there's real problem awaiting to be fixed. 

I believe there is no formal way to raise a bug report for MT5, can anyone who sees this and has the superpowers to raise a bug report please help?

 
It is not a fact that this is a bug
Clearing cache and logs of Tester often helps
 
AIRAT SAFIN #:
It is not a fact that this is a bug
Clearing cache and logs of Tester often helps

Thanks for the suggestion.

If I clear the cache, then it means I lose my optimisation that took hours. This is not acceptable for me, and I think it is something that definitely needs fixing.

I'll try cleaning the logs helps as a workaround, but again if the customer has to remember clearing the logs for the software to function properly, then it is still something that needs fixing.

 
Boogiea999 #:

Thanks for the suggestion.

If I clear the cache, then it means I lose my optimisation that took hours. This is not acceptable for me, and I think it is something that definitely needs fixing.

I'll try cleaning the logs helps as a workaround, but again if the customer has to remember clearing the logs for the software to function properly, then it is still something that needs fixing.

Thanks for the response

I agree that all of these measures are workarounds
But on the other hand developers of top operating systems and browsers also do not have universal solutions for automatically cleaning their systems
And it is hardly a coincidence

As for our automatic trading systems concretely
Maybe it makes sense to look at all this just as sign from high above
The sign that it is time to think about other trading logic because optimization is always insatiable similar to arms race

 
AIRAT SAFIN #:
It is not a fact that this is a bug
Clearing cache and logs of Tester often helps

In my case, it didn't help - although it seems to be fixed with the last update.


Clearing cache, deleting logs, reinstalling metatrader, etc. None of this could prevent it from crashing after doing a few optimizations (memory usage was fine). It's hard to say what caused this bug, but it didn't happen until a few weeks ago. Let's see if it is still a real thing or if it was really fixed with the last patch.

 
Emanuel Cavalcante Amorim Filho #:

In my case, it didn't help - although it seems to be fixed with the last update.


Clearing cache, deleting logs, reinstalling metatrader, etc. None of this could prevent it from crashing after doing a few optimizations (memory usage was fine). It's hard to say what caused this bug, but it didn't happen until a few weeks ago. Let's see if it is still a real thing or if it was really fixed with the last patch.

Clarity 100%
But
The reasons for optimization crashes may be in code itself
I mean the code of program running in optimization mode of Tester
In general Strategy Tester works differently than the MetaEditor Debugger
I am not sure that in optimization mode Strategy Tester will report things like for example dangerous type casts and so on
At this in optimization mode Strategy Tester can use such a number of combinations of parameters that are almost impossible to obtain during usual single runs of Tester or during real-time runs
ESSENCE/SUMMARY
Code review may also be a solution


 

doing any MT5 testing with just 8gb of memory is just crazy and edging into ludicrous. There are so many possible causes of a program freezing -- from just the OS stand point, and 0 way to prove that it is MT issue or an OS issue.

Even if you upgraded to 16gb of memory, the simplest expert such as Expert MAMA, can freeze that 16gb system -- especially if you used optimisation or metaeditor "debugger".

With just 8gb of memory, you can assume that that is the cause of all your heartache. But even 16gb will not get you out of troubles -- at least -- i doubt it. I have 16, and I do not even try optimisations anymore. MT has just got too big and requires a lot more memory than it did just 2 years ago. But it has been like this for awhile now -- just as yourself mention.

 
AIRAT SAFIN #:

Thanks for the response

I agree that all of these measures are workarounds
But on the other hand developers of top operating systems and browsers also do not have universal solutions for automatically cleaning their systems
And it is hardly a coincidence

As for our automatic trading systems concretely
Maybe it makes sense to look at all this just as sign from high above
The sign that it is time to think about other trading logic because optimization is always insatiable similar to arms race


The logs we are talking about are MT5 logs I presume, what does it have to do with development practices of operating systems and browsers? MT5 developers can go their own way if that is the fix and they want to fix it.

I am also puzzled by the second paragraph. Take this as a sign from the heavens to not to use the optimizer instead of asking for a fix??? 

I am looking for some real solutions/workarounds please. As confirmed by Emanuel, the logs is not helping anyway.

 
Michael Charles Schefe #:

doing any MT5 testing with just 8gb of memory is just crazy and edging into ludicrous. There are so many possible causes of a program freezing -- from just the OS stand point, and 0 way to prove that it is MT issue or an OS issue.

Even if you upgraded to 16gb of memory, the simplest expert such as Expert MAMA, can freeze that 16gb system -- especially if you used optimisation or metaeditor "debugger".

With just 8gb of memory, you can assume that that is the cause of all your heartache. But even 16gb will not get you out of troubles -- at least -- i doubt it. I have 16, and I do not even try optimisations anymore. MT has just got too big and requires a lot more memory than it did just 2 years ago. But it has been like this for awhile now -- just as yourself mention.


As I described the optimization finishes without any problem, and it is done on local farm not just one machine so there is no memory issues here. According to the log, the single test also finishes without problem. And memory consumption is nothing large, so 8GB is more than enough. 

The problem is MT5 freezes afterwards. If it was a memory issue, I would have expected to get a low memory warning/error etc. If it was a pointer error, again I would have expected memory violation error etc. but none of those happen. There is no message at all, just a non-responding app.

Reason: