Why break a good old algorithm and replace it with a new one that is 3 times slower?
The genetic tester algorithm has not changed. And statements about 3 times are absolutely unfounded.
The method of work of a cache of previous results changed. And it became much better and more complete than the previous one.
If you need to introduce a new approach to the genetic method, add a new item to the "optimization" tab,
Create a description and techniques for working with it.
Take a look at the links above - all these articles are from our site. The genetic optimizer has been instantly discussed for many years.
Search for "strategy tester", "genetic optimizer" in the search, please.
And take my repeated advice - single recovery factor optimization in genetics is contraindicated. You simply mislead the algorithm with a completely unstable and deceptive(for automatics)"profit/maximum drawdown" ratio, which easily generates huge values. Think about how and why.
We'll correct the drawing of the optimisation graph on Monday.
Have you ever seen volumes in a glass in history? It's impossible to do that without a mould of the glass.
I wouldn't dream of it. I would like if Buy Limit with price 90 and lot 10 would trigger partially, if, say, there was a SELL tick with price 89 and volume 1. It is very sad now. The order would be triggered if there was a BUY tick with the price 90 and lot 1... and in full volume.
settings from https://www.mql5.com/ru/forum/241285/page4#comment_7271206
all tester file caches once removed from disk before tests start
Windows 10, Intel Xeon E5-2630 v4 @ 2.20GHz, 8 cores used for agents
Full pass to end, then terminal restart
with stop at 1000 (approx) passes
Continue after stop
Metatrader 5 build 1755
38 sec, 4780 passes
15 sec, 1105 passes
25 sec, 3487 passes
Metatrader 5 build 1809
33 sec, 5,549 passes
9 sec, 1126 passes
36 sec, 5863 passes
Metatrader 5 build 1755:
Bild 1755 full pass, a total of 8960 variants passed, of which 4780 were actually calculated and 4180 were taken from the cache accumulated in the same session
after which a terminal restart is done to avoid the direct effect of hot caches in memory
2018.04.30 11:20:46.867 Tester Best result 3391.17 produced at generation 22. Next generation 33
2018.04.30 11:20:47.118 Tester genetic calculation is over
2018.04.30 11:20:47.126 Tester 4697 records written to file cache C:\Users\sys\AppData\Roaming\MetaQuotes\Terminal\D0E8209F77C8CF37AD8BF550E51FF075\tester\cache\Moving Average.EURUSD.M5.1.xml
2018.04.30 11:20:47.338 Tester result cache used 4180 times
2018.04.30 11:20:47.338 Tester genetic optimization finished on pass 8960 (of 49644595)
2018.04.30 11:20:47.349 Statistics optimization done in 0 minutes 38 seconds
2018.04.30 11:20:47.349 Statistics local 4780 tasks (100%), remote 0 tasks (0%), cloud 0 tasks (0%)
build 1755 with a stop at 1337 runs. 216 of them are taken from the cache (216 / 1337 = 16% cache hit, not much due to starting randomness and small sample size)
2018.04.30 11:24:15.913 Tester Best result 3254.53 produced at generation 0. Next generation 4
2018.04.30 11:24:16.775 Tester file cache used 19 times
2018.04.30 11:24:16.775 Tester result cache used 216 times
2018.04.30 11:24:16.775 Tester genetic optimization finished on pass 1337 (of 49644595)
2018.04.30 11:24:16.775 Statistics optimization done in 0 minutes 15 seconds
2018.04.30 11:24:16.775 Statistics local 1105 tasks (100%), remote 0 tasks (0%), cloud 0 tasks (0%)
build 1755 with continuation after a stop on Stop, terminal did not restart, continuation.
test continues from stopped point and not from the beginning as in build 1809, then from proposed 10496 passes were taken 3487 and 66+3814 taken from cache (66+3814 / 10496 = 36% cache hit)
2018.04.30 11:26:27.931 Tester Best result 3363.35 produced at generation 15. Next generation 32
2018.04.30 11:26:28.104 Tester genetic calculation is over
2018.04.30 11:26:28.111 Tester 3422 records written to file cache C:\Users\sys\AppData\Roaming\MetaQuotes\Terminal\D0E8209F77C8CF37AD8BF550E51FF075\tester\cache\Moving Average.EURUSD.M5.1.xml
2018.04.30 11:26:28.539 Tester file cache used 66 times
2018.04.30 11:26:28.539 Tester result cache used 3814 times
2018.04.30 11:26:28.539 Tester genetic optimization finished on pass 8704 (of 49644595)
2018.04.30 11:26:28.550 Statistics optimization done in 0 minutes 25 seconds
2018.04.30 11:26:28.550 Statistics local 3487 tasks (100%), remote 0 tasks (0%), cloud 0 tasks (0%)
Metatrader 5 build 1809:
build 1809 full pass, total 8960 passes, of which 5549 were actually counted and 3411 taken from cache hit accumulated in same session
after which restart of terminal is made to avoid direct influence of hot caches in memory.
2018.04.3010:50:58.997 Tester Best result 3391.17 produced at generation 22. Next generation 332018.04.3010:50:59.485 Tester genetic calculation is over
2018.04.3010:50:59.485 Tester result cache used 3411 times
2018.04.3010:50:59.485 Tester genetic optimization finished on pass 8960 (of 49644595)
2018.04.3010:50:59.495 Statistics optimization done in 0 minutes 33 seconds
2018.04.3010:50:59.495 Statistics local 5549 tasks (100%), remote 0 tasks (0%), cloud 0 tasks (0%)
build 1809 with stop at 1348 passes. 222 of them were taken from cache (222 / 1348 = 16% cache hit, not enough because of starting randomness and small sample)
2018.04.3011:00:59.532 Tester Best result 3263.17 produced at generation 0. Next generation 42018.04.3011:00:59.742 Tester result cache used 222 times
2018.04.3011:00:59.742 Tester genetic optimization finished on pass 1348 (of 49644595)
2018.04.3011:00:59.742 Statistics optimization done in 0 minutes 09 seconds
2018.04.3011:00:59.742 Statistics local 1126 tasks (100%), remote 0 tasks (0%), cloud 0 tasks (0%)
build 1809 with continuation after a stop on Stop, terminal did not restart, continuation.
as the test is started from the beginning, from the proposed 10240 passes was made clean 5863 and 4377 were taken from the cache (4377 / 10240 = 42% cache hit)
2018.04.3011:04:28.046 Tester Best result 3391.17 produced at generation 27. Next generation 382018.04.3011:04:28.477 Tester genetic calculation is over
2018.04.3011:04:28.477 Tester result cache used 4377 times
2018.04.3011:04:28.477 Tester genetic optimization finished on pass 10240 (of 49644595)
2018.04.3011:04:28.488 Statistics optimization done in 0 minutes 36 seconds
2018.04.3011:04:28.488 Statistics local 5863 tasks (100%), remote 0 tasks (0%), cloud 0 tasks (0%)
Now the conclusions:
Your statement about "3 times slower" is wrong.
In fact, the tester of build 1809 has become faster through optimization and rewriting of internal processes.
This can be seen in the time of full pass, it's 33 seconds instead of 38 seconds. And 33 seconds for a larger number of passes.
If we recalculate number of passes per second, it turns out: - build 1755 gives 4780 passes / 38 sec = 125 passes per second - build 1809 gives 5549 passes / 33 sec = 168 passes per second
The new scheme of "continuation after stop" genetic optimizer with generation of new population plan is correct.
We are re-evaluating the tester concept and fixing old bad decisions.
Since genetic tests need to be run multiple times to cover the search area more completely, our new model with visualisation of previous runs from the cache allows a better understanding of the mechanics of the genetics workflow.
The new model for displaying optimization results with all the data from previously calculated caches added to the table allows to see the picture more completely. It is possible to compute your strategy in chunks and see full results every time, including previous runs.
It is because of the really large merged results tables that we have disabled the rieltime display of the results list, and show the cumulative result after merging all previous results.
My robot crashes during initialisation, both in Runtime and in Debug. The compilation is fine.
There's a class with static members in the code, probably because of them. Everything was working fine before. But now it reports an error during global initialization.
Removed static members from the class, it runs fine.
My robot crashes during initialisation, both in Runtime and in Debug. The compilation is fine.
There's a class with static members in the code, probably because of them. Everything was working fine before. Now it writes an error during global initialization.
Removed static elements in class, launched normally.
Write your request to Service Desk and attach the Expert Advisor (can be an ex5 file), please.
However, please bring back the "Optimisation" tab - it's impossible to work without operational analytics! Large tables are slow - let's make a filter - show the top 20 for each criterion - it's not so resource-intensive(?), but it will help to see the picture very much. And, those billions of passes - who makes them? They are units with huge capacities - you yourself are talking about the reasonableness of applying genetics, and there are no such portmanteaus there. So more than 10k passes is a rarity.
Sad.
At least it was possible to build a picture on preliminary data while the optimisation was going on.
We are busy with a big performance upgrade of the tester and are redesigning the heavy load modes. Major improvements have been made and new acceleration methods will be implemented soon.
The aisle list window decided to show at the end of the miscalculation, so as not to waste real resources on maintaining, re-sorting and displaying the ever-changing aisle list.
There really was a huge waste of resources and slowdowns. Especially when we're talking hundreds of thousands of rows, millions and tens of millions of passes. There's no reasonable sense in looking at a bunch of preliminary data with your eyes.
We do our own optimization and run tests with 100 million full passes.
It's clear that with such numbers it's out of the question to re-sort in real time and display a table of 2-5-10-50 million values. There is only one option - to gather everything quickly and economically, to final sort it and provide a view of any depth.
Allow the use of cookies to log in to the MQL5.com website.
Please enable the necessary setting in your browser, otherwise you will not be able to log in.
You are signed in. Two-factor authentication has been disabled. Set it up again and scan a new QR code, or continue with two-factor authentication disabled.
Why break a good old algorithm and replace it with a new one that is 3 times slower?
The genetic tester algorithm has not changed. And statements about 3 times are absolutely unfounded.
The method of work of a cache of previous results changed. And it became much better and more complete than the previous one.
If you need to introduce a new approach to the genetic method, add a new item to the "optimization" tab,
Create a description and techniques for working with it.
Take a look at the links above - all these articles are from our site. The genetic optimizer has been instantly discussed for many years.
Search for "strategy tester", "genetic optimizer" in the search, please.
And take my repeated advice - single recovery factor optimization in genetics is contraindicated. You simply mislead the algorithm with a completely unstable and deceptive(for automatics)"profit/maximum drawdown" ratio, which easily generates huge values. Think about how and why.
We'll correct the drawing of the optimisation graph on Monday.
Will there be a partial execution of limit orders, depending on the volumes passing through the market?
Have you ever seen volumes in a glass in history? This is unrealistic to implement without a snapshot of the market.
Have you ever seen volumes in a glass in history? It's impossible to do that without a mould of the glass.
I wouldn't dream of it. I would like if Buy Limit with price 90 and lot 10 would trigger partially, if, say, there was a SELL tick with price 89 and volume 1. It is very sad now. The order would be triggered if there was a BUY tick with the price 90 and lot 1... and in full volume.
The genetic tester algorithm hasn't changed. The claims about 3 times are absolutely unfounded.
What was changed was the way the cache of previous results works. And it became radically better and more complete than the previous one.
Did some comparative testing on the new and old builds of the tester.
Here are the results. Logs and description in post #34.
Here is my test with the suggested conditions:
Metatrader 5 build 1755:
after which a terminal restart is done to avoid the direct effect of hot caches in memory
2018.04.30 11:24:15.913 Tester Best result 3254.53 produced at generation 0. Next generation 4 2018.04.30 11:24:16.775 Tester file cache used 19 times 2018.04.30 11:24:16.775 Tester result cache used 216 times 2018.04.30 11:24:16.775 Tester genetic optimization finished on pass 1337 (of 49644595) 2018.04.30 11:24:16.775 Statistics optimization done in 0 minutes 15 seconds 2018.04.30 11:24:16.775 Statistics local 1105 tasks (100%), remote 0 tasks (0%), cloud 0 tasks (0%)test continues from stopped point and not from the beginning as in build 1809, then from proposed 10496 passes were taken 3487 and 66+3814 taken from cache (66+3814 / 10496 = 36% cache hit)
2018.04.30 11:26:27.931 Tester Best result 3363.35 produced at generation 15. Next generation 32 2018.04.30 11:26:28.104 Tester genetic calculation is over 2018.04.30 11:26:28.111 Tester 3422 records written to file cache C:\Users\sys\AppData\Roaming\MetaQuotes\Terminal\D0E8209F77C8CF37AD8BF550E51FF075\tester\cache\Moving Average.EURUSD.M5.1.xml 2018.04.30 11:26:28.539 Tester file cache used 66 times 2018.04.30 11:26:28.539 Tester result cache used 3814 times 2018.04.30 11:26:28.539 Tester genetic optimization finished on pass 8704 (of 49644595) 2018.04.30 11:26:28.550 Statistics optimization done in 0 minutes 25 seconds 2018.04.30 11:26:28.550 Statistics local 3487 tasks (100%), remote 0 tasks (0%), cloud 0 tasks (0%)Metatrader 5 build 1809:
after which restart of terminal is made to avoid direct influence of hot caches in memory.
as the test is started from the beginning, from the proposed 10240 passes was made clean 5863 and 4377 were taken from the cache (4377 / 10240 = 42% cache hit)
Now the conclusions:
This can be seen in the time of full pass, it's 33 seconds instead of 38 seconds. And 33 seconds for a larger number of passes.
If we recalculate number of passes per second, it turns out:
- build 1755 gives 4780 passes / 38 sec = 125 passes per second
- build 1809 gives 5549 passes / 33 sec = 168 passes per second
We are re-evaluating the tester concept and fixing old bad decisions.
Since genetic tests need to be run multiple times to cover the search area more completely, our new model with visualisation of previous runs from the cache allows a better understanding of the mechanics of the genetics workflow.
The new model for displaying optimization results with all the data from previously calculated caches added to the table allows to see the picture more completely. It is possible to compute your strategy in chunks and see full results every time, including previous runs.
It is because of the really large merged results tables that we have disabled the rieltime display of the results list, and show the cumulative result after merging all previous results.
My robot crashes during initialisation, both in Runtime and in Debug. The compilation is fine.
There's a class with static members in the code, probably because of them. Everything was working fine before. But now it reports an error during global initialization.
Removed static members from the class, it runs fine.
My robot crashes during initialisation, both in Runtime and in Debug. The compilation is fine.
There's a class with static members in the code, probably because of them. Everything was working fine before. Now it writes an error during global initialization.
Removed static elements in class, launched normally.
Write your request to Service Desk and attach the Expert Advisor (can be an ex5 file), please.
All this is great!
However, please bring back the "Optimisation" tab - it's impossible to work without operational analytics! Large tables are slow - let's make a filter - show the top 20 for each criterion - it's not so resource-intensive(?), but it will help to see the picture very much. And, those billions of passes - who makes them? They are units with huge capacities - you yourself are talking about the reasonableness of applying genetics, and there are no such portmanteaus there. So more than 10k passes is a rarity.
Sad.
At least it was possible to build a picture on preliminary data while the optimisation was going on.
We are busy with a big performance upgrade of the tester and are redesigning the heavy load modes. Major improvements have been made and new acceleration methods will be implemented soon.
The aisle list window decided to show at the end of the miscalculation, so as not to waste real resources on maintaining, re-sorting and displaying the ever-changing aisle list.
There really was a huge waste of resources and slowdowns. Especially when we're talking hundreds of thousands of rows, millions and tens of millions of passes. There's no reasonable sense in looking at a bunch of preliminary data with your eyes.
We do our own optimization and run tests with 100 million full passes.
It's clear that with such numbers it's out of the question to re-sort in real time and display a table of 2-5-10-50 million values. There is only one option - to gather everything quickly and economically, to final sort it and provide a view of any depth.
Tell us about it, and that's it.