MetaTrader 4 and MetaTrader 5 Memory and Data Loading Management Comparison - page 7

New comment
 
Icham Aidibe:

My own experience ; as soon as many symbols are needed, data loading becomes problematic ; it requires a strict but not too stressful routine for the memory, not impossible, not easy nor convenient either. 

It's all doable by code.

The bad faith of people is something incredible. Automatic download of data by MT5 is a nice feature, isn't ? Maybe you prefer to grab them manually, that's a choice. Try to do it with MT4 without complete data from the start and see what happens.

 
Icham Aidibe:

btw a parenthesis, on 1958, what is that ? 


A new kind of program is in preparation, it will be Services (beside EAs, Scripts, Indicators). This appears in the log by error (beta version).

Last beta is 1959.

Icham Aidibe:

Just connected with one chart opened, no indicators, nothing. Don't tell me everything's alright, come on ! 

Gonna make a new fresh installation. The installer has been updated and changed too :)

How much "Max bars in chart", how much symbols in Market Watch.

I demonstrated, that in the same condition, MT5 is using less memory than MT4 (and less CPU). Maybe you don't believe me ?

Maybe with this one you will, and the market is now open :


Please stop your subjective complaints.

 

And the last conclusion is this one : when the objective data show the opposite or something different from what people are thinking subjectively and with their a priori, most people prefer to keep their a priori and rejected the data.

This has also be clearly demonstrated in this topic.

 

There is a long answer from Renat on Russian forum:

Forum on trading, automated trading systems and testing trading strategies

New version of MetaTrader 5 build 1930: Floating graphs and .Net libraries in MQL5

Renat Fatkhullin , 2018.12.09 23:30

You have several errors:

  1. It is impossible to test CopyXXX functions from indicators, as only what is given in the indicators is that there is no data integrity control.
  2. Results of operations are not checked.
  3. Monthly data on many instruments in MT4 in a stripped-down form, so it is not always possible to get the requested 500 MN bars

I rewrote the example in the form of a script, reduced the MN to 250 bars, added checks and display information about the environment.

Here are the results on the latest MT4 1150 beta (available on MetaQuotes-Demo) and MT 1959: data in microseconds


MT4
MT5
Cold start 
 5 symbols, 100000 max bars:
    EURUSD
    USDCHF
    USDCAD
    GBPUSD
    USDJPY
Iteration 1 : took 36085 msc for 5 symbols
Iteration 2 : took 155 msc for 5 symbols
...
Iteration 100 : took 138 msc for 5 symbols
Total: 51442 msc


 5 symbols, 100000 max bars:
    EURUSD
    USDCHF
    USDCAD
    GBPUSD
    USDJPY
Iteration 1 : took 1400323 msc for 5 symbols
Iteration 2 : took 12430 msc for 5 symbols
Iteration 3 : took 212 msc for 5 symbols
...
Iteration 100 : took 205 msc for 5 symbols
Total: 1434053 msc

Hot start 
Iteration 1 : took 235 msc for 5 symbols
Iteration 2 : took 138 msc for 5 symbols
...
Iteration 100 : took 136 msc for 5 symbols
Total: 15524 msc


Iteration 1 : took 317 msc for 5 symbols
Iteration 2 : took 204 msc for 5 symbols
...
Iteration 100 : took 201 msc for 5 symbols
Total: 21456 msc

The difference in the cold climb is that the MT5 conducts a full history check and is guaranteed to check with the trading server to download the missing or missing data.

Moreover, the original M1 data is checked, from which other periods are then built. This ensures that the most accurate data in the MT5. In MT4, the most minimal checks and data are taken directly from the file.

In the synchronized data hot mode, the MT4 data is output faster by 6 ms in a request for 45 charts (one iteration of 5 characters for 9 periods), since it has a very simple system for storing charts that are not adapted for working with large volumes. In MT5, we seriously changed the data storage and access mechanisms, which brings additional costs.

Thank you for the test - we will carry out additional optimization in MT5 and try to speed up these places.


Concerning the raised questions on memory consumption in MT4 and MT5:

  • MT4 is a 32-bit program with a lot of restrictions on the amount of data, was developed during the time (2004) of a serious memory deficit
  • MT4 does not have a single base of M1 bars from which other periods are built. Each period is built and lives separately. This reduces memory costs.
  • MT4 has limited multi-threading in internal operations.
  • MT4 has serious limitations on the number of characters and trading environment, including the history

  • MT5 64 bit and is designed to work with huge amounts of data and all processes in it are configured to work and hold the maximum data in memory
  • MT5 always has a M1 basis for the construction of the remaining periods, as well as tick data
  • MT5 has no restrictions on the amount of data processed, a lot of background streams on the resume and data management
  • MT5 tries to use the available memory, rather than trying to shrink to 100-200 mb. Emphasis on maximizing caches
  • MT5 has a much more powerful MQL5 infrastructure inside the terminal
  • MT5 targets computers with 2-4-8-16-32-64 GB of memory, 4-8 or more cores
  • MT5 operates ten times larger data than MT4

  • These are today's data analysis tasks.


Do not think that we develop systems badly or worsen something.

We are focused on optimizing and speeding up data processing. Look in the data catalogs of Metatrader 5 - there are gigabytes and tens of gigabytes of data. And then there will be even more - the amount of data and the quality of analysis grow.

To compare the quality of our optimization and power, look at other C # / Java platforms. This is where braking applications actually operate, operating with ridiculous amounts of data.


 
Alain Verleyen:

A new kind of program is in preparation, it will be Services (beside EAs, Scripts, Indicators). This appears in the log by error (beta version).

Last beta is 1959.

How much "Max bars in chart", how much symbols in Market Watch.

I demonstrated, that in the same condition, MT5 is using less memory than MT4 (and less CPU). Maybe you don't believe me ?

Maybe with this one you will, and the market is now open :


Please stop your subjective complaints.


One thing you did not mention in your experiment is actually the variation of memory usage in MT5. It can be quite big actually.

You can only see the variation when you do multiple of testing in different time like market is busy or market is not busy, or MT5 is busy with writing its own logging files or less busy or indicator itself is busy with writing file or not.

If you are trying to write good amount of data to file after calling data, this is another potential instability inside OnTimer function but I did not have time to got here.

We have not tested dynamic array as well.

Overall, the variation in MT4 is much less even the case of involving file writing or dynamic array or what so ever.

I agree that MT5 can be run smoothly too when there is no heavy data calling or no heavy file writing, etc.

I think you are seeing this case.

As I have mentioned, we are talking about those 90% of average people, who could not modify their virtual memory or could not do fine debugging with Dr. Memory, etc.

So the perception and expectation can be different from tech savvy people.

As long as I see that next version of MT5 can make some improvement on following two issues, I will be really happy.

1. Controlling memory peak problem

2. Removing the Delay in data loading from multiple timeframe and multiple symbols.


My MT5 still show to hit 2 GB memory on start and settle to 700 MB to 800 MB in next five minutes.

In this condition, I am not able to open google chrome or another resource heavy apps.


Anyway, thanks for your hard work. I really appreciated.

When I have time, I might post another testing code with file writing function.

 
Young Ho Seo:


One thing you did not mention in your experiment is actually the variation of memory usage in MT5. It can be quite big actually.

Yes I did. After the peak it remains stable (and lower than MT4) on the given conditions.

You can only see the variation when you do multiple of testing in different time like market is busy or market is not busy, or MT5 is busy with writing its own logging files or less busy or indicator itself is busy with writing file or not.

If you are trying to write good amount of data to file after calling data, this is another potential instability inside OnTimer function as I mentioned in my previous post.

You can test and check all what you want, but please do it in the most correct conditions possible.

We have not tested dynamic array yet, I guess.

Overall, the variation in MT4 is much less even the case of involving file writing or dynamic array or what so ever.

You have to understand that available memory is there to be used, not to remain available eternally. See Renat answer.


I agree that MT5 can be run smoothly too when there is no heavy data calling or no heavy file writing, etc.

I think you are seeing this case.

As I have mentioned, we are talking about those 90% of average people, who could not modify their virtual memory or could not do fine debugging with Dr. Memory, etc.

So the perception and expectation can be different from tech savvy people.

I am talking for my self. I really don't understand why you are keeping talking in the name of other people. Please talk in your name or show that others named you to talk in their name.

As long as I see that next version of MT5 can make some improvement on following two issues, I will be really happy.

1. Controlling memory peak problem

That's a false problem, it exists, but it's normal. Accept it or not, I doubt it will ever change.

2. Removing the Delay in data loading from multiple timeframe and multiple symbols.

It will not change either. See Renat post, it's MT5 architecture and it works well. It's not so easy to use then MT4, that's true, but it's straightforward to manage, you just have to check the returned values and act accordingly. Globally it IMPROVES speed and not the reverse.

My MT5 still show to hit 2 GB memory on start and settle to 700 MB to 800 MB in next five minutes.

In this condition, I am not able to open google chrome or another resource heavy apps.

I understand, try to say to Metaquotes you want them to develop their software for you to be able to open Chrome while you are downloading hundreds(?) symbols data. I don't guarantee your succes.

Anyway, thanks for your hard work. I really appreciated.

When I have time, I might post another testing code with file writing function.

Thanks, you are welcome. Almost...lol

 
Alain Verleyen:
Yes I did. After the peak it remains stable (and lower than MT4) and the given conditions. You can test and check all what you want, but please do it in the most correct conditions possible.
You have to understand that available memory is there to be used, not to remain available eternally. See Renat answer.


I am talking for my self. I really don't understand why you are keeping talking in the name of other people. Please talk in your name or show that others named you to talk in their name. That's a false problem, it exists, but it's normal. Accept it or not, I doubt it will ever change.

It will not change either. See Renat post, it's MT5 architecture and it works well. It's not so easy to use then MT4, that's true, but it's straightforward to manage, you just have to check the returned values and act accordingly. Globally it IMPROVES speed and not the reverse.

I understand, try to say to Metaquotes you want them to develop their software for you to be able to open Chrome while you are downloading hundreds(?) symbols data. I don't guarantee your succes.

Thanks, you are welcome. Almost...lol

Well, it is only straightforward when you code on single currency pairs. If you have to code on 9 timeframe on all symbols in your market watch, this isn't straight forward any more.

Amount of extra code we have to write for this is too much. Especially if your algorithm interrelates two or more currency pairs, the difficulty increases.

As we have seen in MT4 testing, ideally, the data should be loaded in the first iteration. That was beautiful and I will happily lose 1 or 2 second for this reliability but MT5 only returns data after 4 or 5 iterations. :)

I am only talking to MetaQuotes. So do not worry.

 
Young Ho Seo:

Well, it is only straightforward when you code on single currency pairs. If you have to code on 9 timeframe on all symbols in your market watch, this isn't straight forward any more.

Amount of extra code we have to write for this is too much. Especially if your algorithm interrelates two or more currency pairs, the difficulty increases.

As we have seen in MT4 testing, ideally, the data should be loaded in the first iteration. That was beautiful and I will happily lose 1 or 2 second for this reliability. :)

I am only talking to MetaQuotes. So do not worry.

You can hire me, I will code it for you ;-)

It's really hard, so I will ask a very high price, but it will work without problem, you will see

 

What's wrong with using ram? If it does a lot of work it uses either ram or disk. Ram is faster and more durable.

MT5 has a ton of issues, but "using a lot of ram" is not one.

 
kypa:

What's wrong with using ram? If it does a lot of work it uses either ram or disk. Ram is faster and more durable.

MT5 has a ton of issues, but "using a lot of ram" is not one.

Traders are probably thinking RAM is like a Capital, they need to preserve it.
123456789
New comment
Reason: