Discussion of article "Speed Up Calculations with the MQL5 Cloud Network"
What about genetics? There, "100 times" is unattainable. Well, if it's 20-30 times, or even less.
What about genetics? There, "100 times" is unattainable. Well, if it is 20-30 times, or even less.
https://www.mql5.com/ru/forum/4927/page114
stringo 2012.02.02 15:50
At calculation of genetic algorithm the whole next generation (from 64 to 256 calculation jobs) is given to the cloud
In a full search, a stack of 512 jobs is given to each cloud server. Then, when a certain number of jobs are completed, double the number of jobs are added (i.e., a cloud server reports 5 jobs completed, it immediately adds 10 jobs).
so the acceleration of GA is not 20-30, but at least 64 times, at most 256 times.
GA cannot be faster than enumerating a similar number of parameters. The bottleneck of GA is waiting for the slowest agent. And it (on average) occurs from 10240 / 256 to 10240 / 64 times (from 40 to 160 times, based on stringo data). And it is the slowest agent that determines the speed. Besides, in the article Rosh had 4 own local agents -> the speedup limit for GA can be 256 / 4 = 64 times (this is to talk about comparable values), but in life it is much less.
As many noticed, cloud servers detect slow agents automatically and redistribute "slowed" passes to other agents (both in full enumeration and in genetics).
In addition, agents with PR>100 are accepted in genetics, which seriously reduces the consequences of using slow agents.
So, testing conditions - 4 Intel Core i5-2500 @ 3.3 GHz, 4GB RAM, Windows XP 64 bit, terminal x64 bild 581 (PR = 160-180). During testing, was running a browser and watching a film. Used the same conditions as in the article (Moving Avarege Expert Advisor, H1, on OHLC at 1 m. + genetics). Settings:
Test on local kernels:
Test on Cloud:
Without hesitation, we see that Cloud counted 8.7 times faster. That's all. But in fact, the network is even slower because the computation cache was used.
So, the local cores performed 8704 - 3666 = 5038 passes in 30 min. 2 sec = 1802 seconds -> 0.3577 seconds per pass on average.
Cloud performed 8704 - 4455 = 4249 passes in 3 min. 28 sec = 208 seconds -> the waiting time for one pass from Cloud averages 0.049 seconds.
Total, Cloud sped up the computation by a factor of 0.3577 / 0.049 = 7.3 times (!).
My initial assumptions that the figure would probably reach 20 times turned out to be a bit optimistic. And we can't even talk about hundreds.
Yes, I used powerful cores, i.e. I cheated a little. But the fact remains that even those 4249 passes with genetics the network made in 208 seconds and 14040 passes with full search in 164 seconds (waiting time for one pass is 0.0117 seconds). Total, genetic testing of one pass on Cloud is on average slower than brute force (on the example of the EA in question) by 0.049 / 0.0177 = 2.8 times.
I'm not criticising the network - certainly Cloud Network is the best thing that has been done since trading software has existed. And a very necessary thing (although not used much, but that will come with time).
I would like more correctness in advertising slogans. Okay, bygones
If you make tests with the time of one pass less than a second, you must take into account network latency, which is often greater than or equal to the time of a "fast" task. We have made a lot of efforts to get rid of system overheads on network latency by batching tasks. And we have succeeded, although this problem cannot be completely defeated.
Your example fully shows that you have competed in a short-term struggle with a transit time known to be less than the network delay. To really evaluate the power of claud, you need to move away from tasks with execution times less than a second and towards more costly ones.
I've run genetics with a similar task but with a pass time greater than 10 seconds on an i7 processor. I'll post results on completion within the hour.
During 3 min. 28 seconds of network usage, I was charged either 2 or 3 cents (in the terminal - 3, on the site - 2, and 3 frozen). Let it be 3, or even for simplicity, using the network for genetics costs 1 cent for one minute. Total, an hour is 60 cents, 24 hours = $14.4. That sounds very expensive to me. Prices should be reduced at least three times to make it attractive to consumers (many people test EAs, but not everyone is able/willing to pay about $15 per day for Cloud, and if it was $5 or less - there would be more people willing).
Also, I have already figured out for myself how to pre-estimate the cost of optimising any EA (on average, of course). Algorithm for genetics (similarly for brute force):
1) Run Moving Avarage genetics on local cores (+ you can also connect your remote agents); at the end, calculate the waiting time for one pass -> T1
2) Calculate T1 / 0.049 = T2 - how many times the optimisation is accelerated by genetics in Cloud.
3) Run optimisation on local cores (+ you can also connect your remote agents; it should be configured as in step 1) of the required EA and wait for, for example, 100 passes. Calculate the time of one pass (through the log - find the first and last records) -> T3
4) 10240 * T3 / T2 = T4 -> estimated testing time in Cloud.
5) T4 / 60 = testing cost in cents.
(you can also take into account your cores, for this we use T2' = T2 + 1).
This is all an estimate based on Moving Avarage optimisation. Maybe in a month or two, more powerful cores will come online and the numbers will change, as well as the cost.
I think my train of thought is clear
Your example completely shows that you were competing in a short-term fight with a pass time known to be less than the network delay.
Yes, I suspected that, but didn't consider it. Let's wait for your results.
And my previous post, para. 2 will have to be corrected after your tests.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
New article Speed Up Calculations with the MQL5 Cloud Network is published:
How many cores do you have on your home computer? How many computers can you use to optimize a trading strategy? We show here how to use the MQL5 Cloud Network to accelerate calculations by receiving the computing power across the globe with the click of a mouse. The phrase "Time is money" becomes even more topical with each passing year, and we cannot afford to wait for important computations for tens of hours or even days.
Author: MetaQuotes