Question for developers - using all computational cores during optimisation
Why during optimization does Metatrader 5 (latest versions) not use all available calculation cores? In green I marked the cores that perform calculations, while the rest are idle.
They are not idle, they have finished calculations and are waiting for other passes to finish.
They are not idle, but have finished calculations and are waiting for other passes to finish.
Exactly, what are they waiting for when other passes can be done on other cores and it seems to have something to do with it, I have agents on my network, and when it says authorized but the calculation is not finished it just doesn't count further
Exactly, what are they waiting for when other passes can be made on other cores and it seems to have something to do with it, I have agents on my network, and when it says authorized but the calculation is not finished it just doesn't count on
Exactly, what are they waiting for when other passes can be made on other cores and it seems to be related to this hang-up, I have agents in my network, and so when it writes authorized but the calculation is not finished it simply does not count further
Unfortunately, this is how it works in MT5: packets are generated and distributed, but when they are completed in some agents, they are not redistributed. I have written a lot about this already. It's a shame the cores are idle.
Developers don't consider it a priority. They seem to have a lot of cores on their work computers and they are fast, and that evens out the problem. Besides, they are mainly engaged in programming, not in optimization, as we are.
>this is how the MT5 works to distribute packets of jobs. packets are generated, distributed, but when some agents finish, they are not redistributed.
and this is an error ... I had a power outage today and the packets that were distributed to network agents are still unaccounted for ....
>this is how the MT5 works to distribute packets of jobs. packets are generated, distributed, but when some agents finish, they are not redistributed.
and this is an error ... I had a power outage today and packets of jobs that were distributed to agents are not calculated ....
- 2019.11.22
- www.mql5.com
About using physical cores only -
Forum on trading, automated trading systems & strategy testing
Metatester 5 agent manager strategy tester does not see processor threads.
Renat Fatkhullin, 2019.11.27 06:12
Due to the frank lack of memory with too many agents and falling speed of calculations on hypertrading cores, we have decided to limit ourselves to physical cores only when working in the cludes.
Currently there are very costly tasks being run in the cludge, and the vast majority of cludge participants simply do not have enough memory when there are too many agents. For example, 2 gb per agent and 16 agents already require at least 32 gb of memory, even though there is usually 8-16 gb available.
As a result, a lot of passes get screwed up, wasting network and agent load. Most importantly, it slows down the computation because of waiting for results and restarting tasks.
We have long been assessing the approximate resource adequacy of agents before issuing them with tasks, and one of the most effective is to only work on physical cores in the cloud.
Locally, you can use all cores as you can easily manage to disable them.
And this post is about how it all develops further -
Forum on trading, automated trading systems and trading strategy testing
Metatester 5 agent manager strategy tester does not see CPU threads.
Renat Fatkhullin, 2019.11.27 07:18
After the release, we are undertaking a radical overhaul of the resource management of the tester and its agents.
This will primarily affect the efficiency of local agents.
I have no such problems, it sees threads and is not limited to physical cores ... the picture shows the parameters of the processors and memory size ...
about the memory i don't get it at all - swap for what? winds has never been limited to the amount of RAM .... so it's not clear to me where the problem is from the word go
>Do you know that besides SMT2, there are SMT4, SMT8 - when one physical core can do 4 or 8 threads simultaneously, instead of 2 as at Intel? And this is a global trend in high performance computing, these processors are running the most powerful supercomputer on earth. I wouldn't recommend doing with Metatrader what more experienced people do.
The problem is different and it's presented in the picture
for example, there are 20 cores, 19 have finished calculations and are idle, and one needs another 10 calculations, why not do these calculations on free cores/threads - so the solution is not to give each core a bunch of jobs at once, and give only one job, the principle is simple: one thread - one job, finished gets the next job ...
I have no such problems, it sees threads and is not limited to physical cores ... the picture shows the parameters of the processors and memory size ...
about the memory i don't get it at all - swap for what? winds has never been limited to the amount of RAM .... so it's not clear to me where the problem is from the word go
>Do you know that besides SMT2, there are SMT4, SMT8 - when one physical core can do 4 or 8 threads simultaneously, instead of 2 as at Intel? And this is a global trend in high performance computing, these processors are running the most powerful supercomputer on earth. I wouldn't recommend doing with Metatrader what more experienced people do.
The problem is different and it's presented in the picture
The problem is different and it is presented in the picture: you have, for example, 20 cores, 19 have finished calculations and are idle, while one needs another 10 calculations, why don't you make calculations on free cores/threads, so instead of giving each core a gigantic pile of jobs, just one job, simple principle: one thread - one job, when finished you get the next job...
I agree with you, it's a sensible idea. Although I'm not using optimization, but when I was, I always wondered why all threads had finished optimization and one got several more passes. Isn't it possible to redistribute the remaining tasks...
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Why during the optimization metatrader 5 (latest versions) does not use all available computational cores? I marked in green the cores that perform calculations, the rest are idle.