"no memory" when running genetic algorithm tests

[Rafael Caetano Pinto]

I'm testing a multi stocks EA. When I run a single test, everthing works well and the EA consumes 2GB to run the entire history. When I active the generic algorithm, the MT5 was unable to run with "no memory" error.

At the end, the MT5 says: "Tester 24 passes not processed and returned to task queue". But the MT5 does not stop the simulation, the STOP button remains active and when I try to close the MT5, appears a message to stop the simulation before exit.

I disable all cores except one. There is memory enough and the disk is free. But the error persists...

MT5 v5.00 build 2875

[Alain Verleyen]

Code to reproduce the issue ?

[Daniel Weckhuyzen]

It's an issue that has been reported many many times, but until now I've seen no improvements. A lot of people are having this problem. Especially when using tick-data these problems rise very fast. The longer the time-period, the faster these problems arise.

Regards,

Danny

[Alain Verleyen]

Daniel Weckhuyzen:

It's an issue that has been reported many many times, but until now I've seen no improvements. A lot of people are having this problem. Especially when using tick-data these problems rise very fast. The longer the time-period, the faster these problems arise.

Regards,

Danny

Maybe because nobody is doing the needed efforts to report it correctly. I am using the Strategy Tester, backtest and optimization, with or without real ticks and I don't have this issue.

So if someone wants to invest a little time, I am ready to reproduce it, report it and follow it until it's fixed.

[Rafael Caetano Pinto]

I think that it's related with https://www.mql5.com/en/forum/368789, because of the in memory symbols data.

My EA is a multi currency and I'm running on 49 assets. The problem does not happen when I set only around 10 assets. Maybe the MT5 code has any memory threshold to handle with symbol data on tester... But I don't know how to do memory profile in MT5 to dig in this problem.

The EA works fine when running on demo account.

@Alain Verleyen, this is the code.

[Rafael Caetano Pinto]

Today I run this EA in demo account. Works well, except by 2 times when the EA finished because a invalid pointer.

I did it to "solve" the error:

void OnTimer(){
   for(int i=ArraySize(stocks)-1; i>=0; i--){
      StockSymbol *ss = stocks[i];
      if(CheckPointer(ss) == POINTER_INVALID) continue; //I had to add this line to avoid invalid pointer error
      ...//more code here
   }
}

But I was unable to discover how stocks[i] object become invalid. 

I created (and used) the object in OnInit() and I only destroy it in OnDeinit():

StockSymbol* stocks[];

int OnInit(){
   ResetLastError();
   string symbols[];
   if(!ParseSymbols(symbols, InSymbols)) return 4; //fill symbols[] with InSymbols splitted by ';'

   for(int i=ArraySize(symbols)-1; i>=0 && !IsStopped();i--){
      StockSymbol* ss = new StockSymbol(symbols[i]); //I create the object
      ss.Enable(); //I use the object
      
      if(!AddStockSymbol(ss)) return 11; //I put the object into stocks[]
   }
   ...
   return(INIT_SUCCEEDED);
}

void OnDeinit(const int reason){
   for(int i = ArraySize(stocks)-1; i>=0; i--){
      delete stocks[i];
   }
}

bool AddStockSymbol(StockSymbol *ss){
   int size = ArraySize(stocks);
   if(ArrayResize(stocks, size + 1)){
      stocks[size] = ss;
      return true;   
   }
   return false;
}

There is some way of the StockSymbol instance be destroyed by MT5 and be invalid in OnTimer()? Maybe some memory corruption because of excessive use of symbols (49) and indicators (4 to each). The machine memory usage remains below 50% when this EA is running.

[Rafael Caetano Pinto]

Alain Verleyen:

Maybe because nobody is doing the needed efforts to report it correctly. I am using the Strategy Tester, backtest and optimization, with or without real ticks and I don't have this issue.

So if someone wants to invest a little time, I am ready to reproduce it, report it and follow it until it's fixed.

Are there enough information where to do the analysis or I need to provide more? If you need more information, ask me, please.

[Alain Verleyen]

Rafael Caetano Pinto:

Are there enough information where to do the analysis or I need to provide more? If you need more information, ask me, please.

I reproduced the error, I will try to contact MQ to have it fixed.

[Rafael Caetano Pinto]

Alain Verleyen:
I reproduced the error, I will try to contact MQ to have it fixed.

tks a lot

[Rafael Caetano Pinto]

I made another EA. When I run the test the memory issue is worse. The windows reboot 7 times because of MT5 memory drain. 7 times!!!

I reduced the history using only this year and the job was completed with the amount of memory below. The critical part is: the memory used by the strategy tester was not released! This behavior was not found in my multistock EA. This happened with brazilian continuous series data (win$n and wdo$n).

[Rafael Caetano Pinto]

I think that I found the problem:

Alain wrote in other topic (I dont remember what is) that the IndicatorRelease does not works in strategy tester. But is not only the IndicatorRelease but ANY way to release memory (delete also). My last EA created a instance each day (deleting the old instance), but this behavior lead to high memory consumption levels. I changed the EA to use the same instance and removed any delete or release call from the code. After it, the strategy tester problem disappeared.

To all that are having this kind of problems, try this approach and tell here if works to you too.

NOTICE: the memory problem related to download several symbol history data in background remains.