How to release memory after massive CopyRates()?

[terminalstat]

I've noticed that after copying history with CopyRates MT5 keeps copied array in memory.

It's ok for cached access but when I run through lots of symbols my memory ends fast. 

And memory stays occupied until MT5 exits. 

Is there any way to forcibly release memory occupied by CopyRates cache?

[Enrique Dangeroux]

terminalstat:

I've noticed that after copying history with CopyRates MT5 keeps copied array in memory.

It's ok for cached access but when I run through lots of symbols my memory ends fast. 

And memory stays occupied until MT5 exits. 

Is there any way to forcibly release memory occupied by CopyRates cache?

https://www.mql5.com/en/docs/array/arrayfree

[terminalstat]

Sorry, forget to mention that ArrayFree does not affect memory at all.

It seems that itis internal MT5 decision to keep copied data for further access. 

ArrayFree() can't do anithing with it.

[Enrique Dangeroux]

Not sure if you can do anything about the internal workings.

What version Terminal? Maybe this is related https://www.mql5.com/en/forum/326569

[Alain Verleyen]

terminalstat:

Sorry, forget to mention that ArrayFree does not affect memory at all.

It seems that itis internal MT5 decision to keep copied data for further access. 

ArrayFree() can't do anithing with it.

I have serious doubt about your affirmation, can you provide details to prove it ?

I mean there are caches without a doubt, but that it eats all your memory I have never seen that.

[terminalstat]

Alain Verleyen:

I have serious doubt about your affirmation, can you provide details to prove it ?

I mean there are caches without a doubt, but that it eats all your memory I have never seen that.

const int testMaxDepth = 100000;

//+------------------------------------------------------------------+
void OnStart(){

//----check MaxBars limit:
   if(TerminalInfoInteger(TERMINAL_MAXBARS)<testMaxDepth+1000){
      Print("insufficient MaxBars in terminal settings, adjust to ",testMaxDepth+1000,", restart terminal exiting...");
      return;
   }   
   
   ENUM_TIMEFRAMES period=PERIOD_M1;
   
//----going through SYMBOLS:
   for(int s=0;s<SymbolsTotal(false);s++){ 

      const string symbol=SymbolName(s,false);
      
      Print("Scanning ",symbol);

      double hi[],lo[],close[];
      ArraySetAsSeries(hi,true);
      ArraySetAsSeries(lo,true);
      ArraySetAsSeries(close,true);
      if(CopyHigh (symbol,period,0,testMaxDepth,hi)<testMaxDepth)       {Print(GetLastError(),": CopyHigh failed on ",symbol);return;}
      if(CopyLow  (symbol,period,0,testMaxDepth,lo)<testMaxDepth)       {Print(GetLastError(),": CopyLow failed on ",symbol);return;}
      if(CopyClose(symbol,period,0,testMaxDepth,close)<testMaxDepth)    {Print(GetLastError(),": CopyClose failed on ",symbol);return;}
      
      //-some calculations goes here-//

      ArrayFree(hi);
      ArrayFree(lo);
      ArrayFree(close);
   
   }
   
   Print("Program exited, check if memory is released");

     
}

Execution of the above script would occupy about 1 GB of memory (for 35 symbols, the total mount of symbols depends on your broker).

Open task manager before a run and observe memory growth. Memory would stay occupied despite of ArrayFree() and script termination. For a long time. Until terminal exit.

I believe (but definitely not sure) this 1Gb is occupied by copied data, stuck in memory. Next script run would peform very fast. Looks like cache.

Correct me if I'm wrong.

[terminalstat]

found one more struggling:

https://www.mql5.com/en/forum/321480

[Alain Verleyen]

terminalstat:

Execution of the above script would occupy about 1 GB of memory (for 35 symbols, the total mount of symbols depends on your broker).

Open task manager before a run and observe memory growth. Memory would stay occupied despite of ArrayFree() and script termination. For a long time. Until terminal exit.

I believe (but definitely not sure) this 1Gb is occupied by copied data, stuck in memory. Next script run would peform very fast. Looks like cache.

Correct me if I'm wrong.

When you run it, do you have already all data available locally ?

[terminalstat]

I assume yes. Second run are almost instant (and all the rest).

If you quit terminal and run it again - first run slow (hdd data uploaded to memory?), all rest instant.

You can try it/

[terminalstat]

Here I found posible clarification:

https://www.mql5.com/en/docs/series/copyticks

The rate of data return: the terminal stores in the fast access cache 4,096 last ticks for each instrument (65,536 ticks for symbols with a running Market Depth). If requested ticks for the current trading session are beyond the cache, CopyTicks() calls the ticks stored in the terminal memory. These requests require more time for execution. The slowest requests are those requesting ticks for other days, since the data is read from the disk in this case.

But no information how to clear that cached data out. I have about 1000 CFD symbols to scan. In various timeframes for each. A lot of memory required)

[Alain Verleyen]

terminalstat:

I assume yes. Second run are almost instant (and all the rest).

If you quit terminal and run it again - first run slow (hdd data uploaded to memory?), all rest instant.

You can try it/

I am trying it. I will probably report my findings (if any) later.

I have a question, let's say you see well and MT5 cache data for a long while. What is the actual problem with that, do you have a real issue or is this just a matter of knowledge and understanding ? As it seems to me using memory as cache is a good thing, provided it doesn't impact other tasks of course.