MT5 Strategy Tester improvement request: Task allocation is currently not optimal
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Hi,
I've noticed this ever since I switched to MT5 and multi-agent testing around 2 years ago.
When testing with all local agents (I have not yet used cloud testing) the task scheduler does not distribute tasks optimally - particularly when getting "near" the end of testing.
For genetic optimisation it initially allocates about 90% of the first generation's tasks (which is good), but then when the first agent finishes its tasks it gets basically the entire remaining 10% of tasks, which means that when the other agents finish they sit idle waiting for a single agent to crunch through many tasks, when they could be done many times faster by being redistributed to idle agents. Thus the first generation takes solidly longer than it should.
This gets even worse for forward testing, as my current run shows. After the genetic portion of the test, the forward testing began - & it looks like it is forward testing ALL passes done on the backtest period, not just the 25% or 40% best scored that the documentation states should be tested. What this has meant is that 1 agent has now finished its assigned tasks, & others have hundreds left to process - 1 agent has around 900 to do, while the others have fewer. This is one reason why I generally don't use forward testing.
What would be good is that the scheduler notices when an agent has finished its tasks, & reassigns tasks from other agents to the idle agent - particularly the agent with the most tasks left. That way, the testing would finish significantly faster, with ALL agents sharing the load until there are no more tasks to process.
I'm surprised this hasn't been requested before now, or perhaps not even noticed by the MT5 team, & I have found no settings where I can change this behaviour.
I've attached an image showing the idle agent, with over 2000 tasks left to process - which will take around 18 hours. This time would be much reduced with the addition of the currently single idle agent (& multiple idle agents the closer it gets to the end of testing) - though given the major difference in number left to process (between 150 & 900) it could be more than a day before the testing finishes, where it should be under 16 hours with all agents utilised.
For those wondering why I'm not using more agents given my CPU, the reason is simple: 5 agents has already nearly maxed out my CPU-RAM bandwidth (testing my currency strength EA, so accessing data from 28 currency pairs simultaneously), & running more agents actually returns results more slowly than with just 5 agents. I'm considering an upgrade to 128GB (dual channel 4x32GB DDR4-3200, up from dual channel 2x32GB DDR4-3200) to increase CPU-RAM bandwidth & hence number of agents that can be used in my testing. My CPU will happily run 9 agents for a single chart EA optimisation & rip through a 12000 pass optimisation (using M1 OHLC mode) on a 5 year period in 3 hours or less, but currency strength EA's looking at 28 pairs are far more demanding.
Being able to tweak some of the Strategy Tester settings (especially those used in genetic testing) would be nice to have as well, since it can get hyper-focused on narrow ranges as more generations go by, with almost no passes using even moderately randomised settings.
Thanks,
Brendan.