Optimiser Errors "no disk space in ticks generating function" - page 3
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
my issue was mostly related to my testing account being in AUD currency, so tester had to use 3 different pairs. But once i changed my currency to one that was included in the symbol, ie EUR because I was testing EURGBP, then, my memory usage dropped to 250 - 600mb per core/thread. And all was good from there on.
NOTE: i have 64gb memory, cpu is 8core/16thread, with 4tb of drive space. so, yes, my issue was only due to memory, which was already clear to me from similar issues in past systems with smaller memory(s). Increasing the pagefile resulted in a longer period before the error happened, but it only delayed the inevitable, as it always happened, no matter how big i allowed windows to expand the pagefile to. But now I am back to being able to disable the pagefile completely and have all 16 cores(threads) running 24/7 and able to run computer relatively normally as my memory usage is less than 70% even with a cache program running with mt5; and 3 other mts running 1 - 2gbs each with media player running almost 100% normally; only flickering when each video starts.my issue was mostly related to my testing account being in AUD currency, so tester had to use 3 different pairs. But once i changed my currency to one that was included in the symbol, ie EUR because I was testing EURGBP, then, my memory usage dropped to 250 - 600mb per core/thread. And all was good from there on.
NOTE: i have 64gb memory, cpu is 8core/16thread, with 4tb of drive space. so, yes, my issue was only due to memory, which was already clear to me from similar issues in past systems with smaller memory(s). Increasing the pagefile resulted in a longer period before the error happened, but it only delayed the inevitable, as it always happened, no matter how big i allowed windows to expand the pagefile to. But now I am back to being able to disable the pagefile completely and have all 16 cores(threads) running 24/7 and able to run computer relatively normally as my memory usage is less than 70% even with a cache program running with mt5; and 3 other mts running 1 - 2gbs each with media player running almost 100% normally; only flickering when each video starts.mine were custom symbols but it doesn't matter what currency makes up the pair it always throws it at 4 or more symbols so my next pc Im prob going to buy a used HP Z440 and juice it up with 128GB of RAM which should allow me to do 28 different pairs in a backtest.
Couldn't the reason be simply a poorly coded EA? I haven't used the strategy tester much, but I can say this much that my custom-built optimizers even work inside the mql5 optimizer. At least they loads the necessary data into memory as written in the program. I also got errors when updating the data and haven't had time to delve into it yet. But such things have not happened that the memory or disk could be overloaded...
Do memory release functions really not work in the optimizer correctly if they are properly coded into the EA? ...and the optimizer starts writing everything to disk when the memory is full? However, I still believe that it is bad code that causes such problems.
At this point, it is logical that the tick data itself is already on disk after some tests, so the problem here must be that the EA is unable to process the data properly and keeps everything in memory due to poor memory management, or is it really just the behavior of the optimizer?
Couldn't the reason be simply a poorly coded EA? I haven't used the strategy tester much, but I can say this much that my custom-built optimizers even work inside the mql5 optimizer. At least they loads the necessary data into memory as written in the program. I also got errors when updating the data and haven't had time to delve into it yet. Do memory release functions really not work in the optimizer if they are properly coded into the EA? ...and the optimizer starts writing everything to disk when the memory is full?
good questions, but the issue is the norm for all eas i have used in optimiser. if those memory functions do work, then, it would be apparent that you need to have specific coding for optimiser, just like you often need specific coding for strategy tester.
If you prove or disprove any of the questions and statements you said, then, please add your findings to this thread. I am sure we would all like to know.good questions, but the issue is the norm for all eas i have used in optimiser. if those memory functions do work, then, it would be apparent that you need to have specific coding for optimiser, just like you often need specific coding for strategy tester.
I can't speak more specifically here because I haven't used traditional indicators and their buffers, I just deal with the data flow "manually", but as far as I've seen superficially, many indicator coders don't know much about data processing and simply spread the buffer into memory, without really managing it. This could theoretically be the key to the problem... but that's just a my first theory...
At this point, if I have time, I can analyze the codes a bit. You can send me the codes of some problematic EAs. Out of sportsmanship, I could study them a bit. Maybe we can get better that way?
I'm just curious where these random problems come from, are they caused by the optimizer/strategy tester code or the EA code, because I got some weird behaviors in the mql5 strategy tester myself.
In theory, the optimizer should not store anything except the optimized parameters and their results in memory - could some MQL5 developer answer here how things are and why?
I can't speak more specifically here because I haven't used traditional indicators and their buffers, I just deal with the data flow "manually", but as far as I've seen superficially, many indicator coders don't know much about data processing and simply spread the buffer into memory, without really managing it. This could theoretically be the key to the problem... but that's just a my first theory...
At this point, if I have time, I can analyze the codes a bit. You can send me the codes of some problematic EAs. Out of sportsmanship, I could study them a bit. Maybe we can get better that way?
I'm just curious where these random problems come from, are they caused by the optimizer code or the EA code, because I got some weird behaviors in the mql5 strategy tester myself.
i totally agree with your hypothesis, however, most indicators do not need further memory management; however, it makes theoretical sense that it would be much more likely to be required in the instances where optimiser would be expected to be used in testing such indicators.
i totally agree with your hypothesis, however, most indicators do not need further memory management; however, it makes theoretical sense that it would be much more likely to be required in the instances where optimiser would be expected to be used in testing such indicators.
I don't know how and if the higher-level automatic memory management functions like garbage collection work in the tester and also in mql5 general. Are there automatic memory management at all? Anyway, mql5 has a built-in functionality for higher-level memory management. I assume there's a reason for this? If this is not followed, then maybe problems will start to happen? In live doesn't understand this, because in live the memory fills up slowly, but when testing strategies, the data starts to accumulate quickly. But this is just my hypothesis...
Couldn't the reason be simply a poorly coded EA? I haven't used the strategy tester much, but I can say this much that my custom-built optimizers even work inside the mql5 optimizer. At least they loads the necessary data into memory as written in the program. I also got errors when updating the data and haven't had time to delve into it yet. But such things have not happened that the memory or disk could be overloaded...
Do memory release functions really not work in the optimizer correctly if they are properly coded into the EA? ...and the optimizer starts writing everything to disk when the memory is full? However, I still believe that it is bad code that causes such problems.
At this point, it is logical that the tick data itself is already on disk after some tests, so the problem here must be that the EA is unable to process the data properly and keeps everything in memory due to poor memory management, or is it really just the behavior of the optimizer?
how do you build "custom optimizers" or do you mean custom performance metrics? also the 1GB of RAM used for me is per core and each core is doing 3 symbols across 3 years idk if my other post explained it well. so my 16GB of RAM with 11/12 cores 11*1GB = 11GB used just for mt5 and the rest is being used by windows to run bloatware or something. I've also deleted all the temporary files and it still maxes out at the same levels each time so thats not really an issue for me
I don't know how and if the higher-level functions like garbage collection work in the tester and also in mql5 general. Are there automatic memory management at all? Anyway, mql5 has a built-in functionality for higher-level memory management. I assume there's a reason for this. If this is not followed, then maybe problems will start to happen? In live doesn't understand this, because in live the memory fills up slowly, but when testing strategies, the data starts to accumulate quickly. But this is just my hypothesis...
all you said makes complete sense to me. And as for "higher memory functions". i do remember seeing memory mentioned many times in more advanced coding threads.
how do you build "custom optimizers" or do you mean custom performance metrics? also the 1GB of RAM used for me is per core and each core is doing 3 symbols across 3 years idk if my other post explained it well. so my 16GB of RAM with 11/12 cores 11*1GB = 11GB used just for mt5 and the rest is being used by windows to run bloatware or something. I've also deleted all the temporary files and it still maxes out at the same levels each time so thats not really an issue for me
I mean the result, it doesn't matter if this optimizer is a visually visible GUI that we have to run or if we get the same optimized values by processing the data stream...
I can't comment - if 1 GB per core stably maintains its value without changing the hard drive capacity, everything would seem to be maybe fine? Although 1 GB per instrument seems like a lot to me. But then the strategy tester is built that way and it would be wrong if it accumulated over each time massively. So, I believe that the main thing is that the data volume remains uniform when using the strategy tester and it does not grow massively. In this case, everything seems to be under control. There is no point in picking at this point why so much memory is used, the main thing is that the memory usage does not grow over time.
However, it would be interesting to test with programs optimized for small memory usage and speed to see if the memory usage in the optimizer is still that high. I'll try it out soon and let you know. 1GB per instrument seems still crazy for me...
It could also be that the tester itself stores 1 GB of data in memory at a time for one instrument because it would also be unthinkable if too little data was loaded into memory and disk I/O was constantly being used to update the data.
But that's also a theory... Need to test...
It's a shame that the MQL5 developers themselves never answer users' questions and we have to discover everything ourselves...