Metatrader 5 - Takes alot of memory when cycling a chart through symbols - page 4

[gr101]

Carl Schreiber:

Have you tried to setTERMINAL_MAXBARS with TerminalInfoInteger(..) to a reasonable small value?

Beside that you have:

TERMINAL_MEMORY_PHYSICAL	Physical memory in the system, Mb	int
TERMINAL_MEMORY_TOTAL	Memory available to the process of the terminal , Mb	int
TERMINAL_MEMORY_AVAILABLE	Free memory of the terminal process, Mb	int
TERMINAL_MEMORY_USED	Memory used by the terminal , Mb	int

to check where's the crux of the matter.

I set the TERMINAL_MAXBARS to 500 (really low) but it's still loading more bars (imagine that's from file?) and the memory is still creeping up.

Have started logging the memory counters listed above will have some feedback on that soon.

Ok got some feedback and it looks a little strange. Below are two logs "MetaTrader Log" & "PowerShell Log" they are two logs I'm running that are logging the memory usage. (PowerShell script is earlier in this thread, I updated it to match what's in Task Manager). Both of these logs are the first and last times when I did the last run.

Points of interest:

1. The TERMINAL_MEMORY_PHYSICAL and TERMINAL_MEMORY_TOTAL remain the same. These values are identical in every row all the way through the log.
   
2. The TERMINAL_MEMORY_AVAILABLE has decreased from 13902 to 13806
3. The TERMINAL_MEMORY_USED has increased from 432 to 528
4. The number of Kilo Bytes used by the system went from 128 300 K (128.3 Mb) on start up to 215 488 K (215.488 Mb) over the duration of the testing

##### MetaTrader Log
##### This was logged by my logger in Metatrader. It outputs all the memory

TIME	TERMINAL_MEMORY_PHYSICAL	TERMINAL_MEMORY_TOTAL	TERMINAL_MEMORY_AVAILABLE	TERMINAL_MEMORY_USED
2016.04.29 17:31:58	7167	14334	13902	432
2016.04.29 18:04:38	7167	14334	13806	528

#### PowerShell Log
#### This was logged by the PowerShell script

04/29/2016 17:31:32 Kilo Bytes in use  128 300 K
04/29/2016 18:05:08 Kilo Bytes in use  215 488 K

[Carl Schreiber]

gr101:

I set the TERMINAL_MAXBARS to 500 (really low) but it's still loading more bars (imagine that's from file?) and the memory is still creeping up.

Have started logging the memory counters listed above will have some feedback on that soon.

Ok got some feedback and it looks a little strange. Below are two logs "MetaTrader Log" & "PowerShell Log" they are two logs I'm running that are logging the memory usage. (PowerShell script is earlier in this thread, I updated it to match what's in Task Manager). Both of these logs are the first and last times when I did the last run.

Points of interest:

1. The TERMINAL_MEMORY_PHYSICAL and TERMINAL_MEMORY_TOTAL remain the same. These values are identical in every row all the way through the log.
   
2. The TERMINAL_MEMORY_AVAILABLE has decreased from 13902 to 13806
3. The TERMINAL_MEMORY_USED has increased from 432 to 528
4. The number of Kilo Bytes used by the system went from 128 300 K (128.3 Mb) on start up to 215 488 K (215.488 Mb) over the duration of the testing

##### MetaTrader Log
##### This was logged by my logger in Metatrader. It outputs all the memory

TIME	TERMINAL_MEMORY_PHYSICAL	TERMINAL_MEMORY_TOTAL	TERMINAL_MEMORY_AVAILABLE	TERMINAL_MEMORY_USED
2016.04.29 17:31:58	7167	14334	13902	432
2016.04.29 18:04:38	7167	14334	13806	528

#### PowerShell Log
#### This was logged by the PowerShell script

04/29/2016 17:31:32 Kilo Bytes in use  128 300 K
04/29/2016 18:05:08 Kilo Bytes in use  215 488 K

I (notebook, 8GB ram, Win7-64) have equivalent ram usage for each of my 4 mt4.

I think it's nothing you have to worry about!

How many GB are still free/available?

[gr101]

Carl Schreiber:

I (notebook, 8GB ram, Win7-64) have equivalent ram usage for each of my 4 mt4.

I think it's nothing you have to worry about!

How many GB are still free/available?

I've got loads of RAM at the levels above in the graphs. The issue is when I leave it running it slowly pushes the RAM usage up.

So after about an hour or two it starts maxing out. 

I'm currently running the EA on Server 2012, 7GB RAM, 4 Cores (not sure what they are), 64 bit.

[Carl Schreiber]

gr101:

I've got loads of RAM at the levels above in the graphs. The issue is when I leave it running it slowly pushes the RAM usage up.

So after about an hour or two it starts maxing out. 

I'm currently running the EA on Server 2012, 7GB RAM, 4 Cores (not sure what they are), 64 bit.

I once learned it seems to be a feature of Win-Server keeping as much as possible in the memory.

I restarted the terminals regularly but I would be curious whether this would cause a crash or not!

[gr101]

Carl Schreiber:

I once learned it seems to be a feature of Win-Server keeping as much as possible in the memory.

I restarted the terminals regularly but I would be curious whether this would cause a crash or not!

Interesting... although it does seem to be something to do with loading symbols into and out of the Market Watch.

The terminal does not crash it just starts to run really slow, unusabley slow.

[Carl Schreiber]

gr101:

Interesting... although it does seem to be something to do with loading symbols into and out of the Market Watch.

The terminal does not crash it just starts to run really slow, unusabley slow.

How do you check this? Maybe the server is not set up as fast machine for graphics but only fast math. calculations and quick web responses?

[gr101]

It's a standard install of Server 2012. However my theory is that it does not matter how it's setup. A memory leak will, over time, always take up all the space on the machine.

So it doesn't matter how fast or how much RAM the box has, if it has a leak then it'll hit the limit at some point.

[Rogerio Figurelli]

Just to contribute with my 2 cents on this topic.

There are a lot of good ideas here about memory leak and increase, but regarding the topic and the root cause my considerations are:

- the published source code doesn't check several relevant function returns, for instance SymbolSelect() return.

- for/while loops based on functions return, like SymbolsTotal() could have some limit or protection against some wrong value function return. 

After stress/change the code and test again, maybe we can find and comprove the root cause of the problem.

[Marco vd Heijden]

I think Rogerio Figurelli is correct if many function calls are not returned the allocated memory will not be freed.

Maybe you can decrease stacksize to see if it gives stack overflow error.

Stack

In every MQL4 program, a special memory area called stack is allocated for storing local function variables that are created automatically. One stack is allocated for all functions. The default stack size is 256 kb, the stack size can be managed using the #property stacksize compiler directive.

Static local variables are stored in the same place where other static and global variables are stored - in a special memory area, which exists separately from the stack. Dynamically created variables also use a memory area separate from the stack.

With each function call, a place on the stack is allocated for internal non-static variables. After exiting the function, the memory is available for use again.

If from the first function the second one is called, then the second function occupies the required size from the remaining stack memory for its variables. Thus, when using included functions, stack memory will be sequentially occupied for each function. This may lead to a shortage of memory during one of the function calls, such a situation is called stack overflow.

Therefore, for large local data you should better use dynamic memory - when entering a function, allocate the memory, which is required for local needs, in the system (new, ArrayResize()), and when exiting the function, release the memory (delete, ArrayFree()).

See also

Data Types, Encapsulation and Extensibility of Types,Initialization of Variables, Visibility Scope and Lifetime of Variables, Creating and Deleting Objects

[gr101]

Good points.

Will give this a try sometime and feedback here with results.

Thanks