MT5 Strategy Tester freezing at 99.8%

 

I have a relatively simple EA that I'm optimizing and when I run tick data the tester freezes at 99.8% complete and produces no error messages but will just sit there for hours until I stop the optimization.  The peculiar behavior is that my agents can process 1 test just fine, but if it has to do more than 1 pass it will fail at the 99.8%.  I'm running tick data on the M10 timeframe from 1/1/2020 to present, using 1/3 of the data for forward testing.  My main PC is a Ryzen 7 3700X 8 core 16 thread processor, with 32 gigs or ram.  I also have a Dell R730 server with 24 cores, 48 threads and 160 GB ram.  The tests fail on each machine in the exact same way.  The server is using  the 4120 build, and my PC is on 4105.  Using 1M OHLC data obviously produces 0 errors. 


Thoughts on how to troubleshoot this so I can actually optimize with tick data? 

 
Your topic has been moved to the section: Expert Advisors and Automated Trading
Please consider which section is most appropriate — https://www.mql5.com/en/forum/172166/page6#comment_49114893
 
chrisaaronmurphy:

I have a relatively simple EA that I'm optimizing and when I run tick data the tester freezes at 99.8% complete and produces no error messages but will just sit there for hours until I stop the optimization.  The peculiar behavior is that my agents can process 1 test just fine, but if it has to do more than 1 pass it will fail at the 99.8%.  I'm running tick data on the M10 timeframe from 1/1/2020 to present, using 1/3 of the data for forward testing.  My main PC is a Ryzen 7 3700X 8 core 16 thread processor, with 32 gigs or ram.  I also have a Dell R730 server with 24 cores, 48 threads and 160 GB ram.  The tests fail on each machine in the exact same way.  The server is using  the 4120 build, and my PC is on 4105.  Using 1M OHLC data obviously produces 0 errors. 


Thoughts on how to troubleshoot this so I can actually optimize with tick data? 

It needs more details but as a rough answer:

- start with one or two very clear inputs to optimize. very clear means you are sure values of that input in all cases will not cause an infinite or almost infinite loop.  this can be happen for inputs that make very high number of trades like 1B. Also for this simple test, do not choose inputs that are effective when another input is true or false.

- reduce number of values for that one or two inputs. for example  if you want to check lot size from 0.01 to 1.0 do not set step to 0.01 for this simple test. make it 0.1 or 0.25

- during optimization, check if there is an Agent that can not complete its jobs. like if you have 8 core and Tasks/Passed is 10/0 for one of them and 10/10 for others then some how one or more input values cause what I mentioned as infinite or big loops. correct them again according above items and limit them more. 

- try to select another very simple input to test if it happens again.

- you may need add testing codes to OnTester functions (OnTesterInit(), ...) to analyze your Testing/Optimization process better.

 
Mahdi Ebrahimzadeh #:

It needs more details but as a rough answer:

- start with one or two very clear inputs to optimize. very clear means you are sure values of that input in all cases will not cause an infinite or almost infinite loop.  this can be happen for inputs that make very high number of trades like 1B. Also for this simple test, do not choose inputs that are effective when another input is true or false.

- reduce number of values for that one or two inputs. for example  if you want to check lot size from 0.01 to 1.0 do not set step to 0.01 for this simple test. make it 0.1 or 0.25

- during optimization, check if there is an Agent that can not complete its jobs. like if you have 8 core and Tasks/Passed is 10/0 for one of them and 10/10 for others then some how one or more input values cause what I mentioned as infinite or big loops. correct them again according above items and limit them more. 

- try to select another very simple input to test if it happens again.

- you may need add testing codes to OnTester functions (OnTesterInit(), ...) to analyze your Testing/Optimization process better.

Appreciate the feedback here.  I will try to provide more information as I continue to optimize. 

 
Mahdi Ebrahimzadeh #:

It needs more details but as a rough answer:

- start with one or two very clear inputs to optimize. very clear means you are sure values of that input in all cases will not cause an infinite or almost infinite loop.  this can be happen for inputs that make very high number of trades like 1B. Also for this simple test, do not choose inputs that are effective when another input is true or false.

- reduce number of values for that one or two inputs. for example  if you want to check lot size from 0.01 to 1.0 do not set step to 0.01 for this simple test. make it 0.1 or 0.25

- during optimization, check if there is an Agent that can not complete its jobs. like if you have 8 core and Tasks/Passed is 10/0 for one of them and 10/10 for others then some how one or more input values cause what I mentioned as infinite or big loops. correct them again according above items and limit them more. 

- try to select another very simple input to test if it happens again.

- you may need add testing codes to OnTester functions (OnTesterInit(), ...) to analyze your Testing/Optimization process better.

Can we get inputs that freeze optimization? I want to find the inputs that freeze the optimization and debug in single test.
 
Delete the contents of Tester/Cache. Delete the old tester logs as well

My cache folder was 6GB. Deleting this resolved these issues, and the test finishes very quickly now.