MT5 terminal updated today and the "Optimisation" window does not show up during the test - page 5

 
Texnolog:

Why break a good old algorithm and replace it with a new one that is 3 times slower?

The genetic tester algorithm has not changed. And statements about 3 times are absolutely unfounded.

The method of work of a cache of previous results changed. And it became much better and more complete than the previous one.

If you need to introduce a new approach to the genetic method, add a new item to the "optimization" tab,

Create a description and techniques for working with it.

Take a look at the links above - all these articles are from our site. The genetic optimizer has been instantly discussed for many years.

Search for "strategy tester", "genetic optimizer" in the search, please.

And take my repeated advice - single recovery factor optimization in genetics is contraindicated. You simply mislead the algorithm with a completely unstable and deceptive(for automatics)"profit/maximum drawdown" ratio, which easily generates huge values. Think about how and why.


We'll correct the drawing of the optimisation graph on Monday.

 
Will there be partial execution of limit orders, depending on the volume of the market?
 
pivomoe:
Will there be a partial execution of limit orders, depending on the volumes passing through the market?

Have you ever seen volumes in a glass in history? This is unrealistic to implement without a snapshot of the market.

 
Aleksey Vyazmikin:

Have you ever seen volumes in a glass in history? It's impossible to do that without a mould of the glass.

I wouldn't dream of it. I would like if Buy Limit with price 90 and lot 10 would trigger partially, if, say, there was a SELL tick with price 89 and volume 1. It is very sad now. The order would be triggered if there was a BUY tick with the price 90 and lot 1... and in full volume.

 
Renat Fatkhullin:

The genetic tester algorithm hasn't changed. The claims about 3 times are absolutely unfounded.

What was changed was the way the cache of previous results works. And it became radically better and more complete than the previous one.


Did some comparative testing on the new and old builds of the tester.

Here are the results. Logs and description in post #34.

11

 

Here is my test with the suggested conditions:

  • settings from https://www.mql5.com/ru/forum/241285/page4#comment_7271206
  • all tester file caches once removed from disk before tests start
  • Windows 10, Intel Xeon E5-2630 v4 @ 2.20GHz, 8 cores used for agents

Full pass to end, then terminal restart
with stop at 1000 (approx) passes
Continue after stop
Metatrader 5 build 1755
38 sec, 4780 passes
15 sec, 1105 passes
25 sec, 3487 passes
Metatrader 5 build 180933 sec, 5,549 passes
9 sec, 1126 passes
36 sec, 5863 passes


Metatrader 5 build 1755:

  • Bild 1755 full pass, a total of 8960 variants passed, of which 4780 were actually calculated and 4180 were taken from the cache accumulated in the same session

    after which a terminal restart is done to avoid the direct effect of hot caches in memory
    2018.04.30 11:20:46.867	Tester	Best result 3391.17 produced at generation 22. Next generation 33
    2018.04.30 11:20:47.118	Tester	genetic calculation is over
    2018.04.30 11:20:47.126	Tester	4697 records written to file cache C:\Users\sys\AppData\Roaming\MetaQuotes\Terminal\D0E8209F77C8CF37AD8BF550E51FF075\tester\cache\Moving Average.EURUSD.M5.1.xml
    2018.04.30 11:20:47.338	Tester	result cache used 4180 times
    2018.04.30 11:20:47.338	Tester	genetic optimization finished on pass 8960 (of 49644595)
    2018.04.30 11:20:47.349	Statistics	optimization done in 0 minutes 38 seconds
    2018.04.30 11:20:47.349	Statistics	local 4780 tasks (100%), remote 0 tasks (0%), cloud 0 tasks (0%)
    

  • build 1755 with a stop at 1337 runs. 216 of them are taken from the cache (216 / 1337 = 16% cache hit, not much due to starting randomness and small sample size)
    2018.04.30 11:24:15.913	Tester	Best result 3254.53 produced at generation 0. Next generation 4
    2018.04.30 11:24:16.775	Tester	file cache used 19 times
    2018.04.30 11:24:16.775	Tester	result cache used 216 times
    2018.04.30 11:24:16.775	Tester	genetic optimization finished on pass 1337 (of 49644595)
    2018.04.30 11:24:16.775	Statistics	optimization done in 0 minutes 15 seconds
    2018.04.30 11:24:16.775	Statistics	local 1105 tasks (100%), remote 0 tasks (0%), cloud 0 tasks (0%)
    
  • build 1755 with continuation after a stop on Stop, terminal did not restart, continuation.

    test continues from stopped point and not from the beginning as in build 1809, then from proposed 10496 passes were taken 3487 and 66+3814 taken from cache (66+3814 / 10496 = 36% cache hit)
    2018.04.30 11:26:27.931	Tester	Best result 3363.35 produced at generation 15. Next generation 32
    2018.04.30 11:26:28.104	Tester	genetic calculation is over
    2018.04.30 11:26:28.111	Tester	3422 records written to file cache C:\Users\sys\AppData\Roaming\MetaQuotes\Terminal\D0E8209F77C8CF37AD8BF550E51FF075\tester\cache\Moving Average.EURUSD.M5.1.xml
    2018.04.30 11:26:28.539	Tester	file cache used 66 times
    2018.04.30 11:26:28.539	Tester	result cache used 3814 times
    2018.04.30 11:26:28.539	Tester	genetic optimization finished on pass 8704 (of 49644595)
    2018.04.30 11:26:28.550	Statistics	optimization done in 0 minutes 25 seconds
    2018.04.30 11:26:28.550	Statistics	local 3487 tasks (100%), remote 0 tasks (0%), cloud 0 tasks (0%)
    





Metatrader 5 build 1809:

  • build 1809 full pass, total 8960 passes, of which 5549 were actually counted and 3411 taken from cache hit accumulated in same session

    after which restart of terminal is made to avoid direct influence of hot caches in memory.
    2018.04.30 10:50:58.997 Tester  Best result 3391.17 produced at generation 22. Next generation 33
    2018.04.30 10:50:59.485 Tester  genetic calculation is over
    2018.04.30 10:50:59.485 Tester  result cache used 3411 times
    2018.04.30 10:50:59.485 Tester  genetic optimization finished on pass 8960 (of 49644595)
    2018.04.30 10:50:59.495 Statistics      optimization done in 0 minutes 33 seconds
    2018.04.30 10:50:59.495 Statistics      local 5549 tasks (100%), remote 0 tasks (0%), cloud 0 tasks (0%)
    

  • build 1809 with stop at 1348 passes. 222 of them were taken from cache (222 / 1348 = 16% cache hit, not enough because of starting randomness and small sample)
    2018.04.30 11:00:59.532 Tester  Best result 3263.17 produced at generation 0. Next generation 4
    2018.04.30 11:00:59.742 Tester  result cache used 222 times
    2018.04.30 11:00:59.742 Tester  genetic optimization finished on pass 1348 (of 49644595)
    2018.04.30 11:00:59.742 Statistics      optimization done in 0 minutes 09 seconds
    2018.04.30 11:00:59.742 Statistics      local 1126 tasks (100%), remote 0 tasks (0%), cloud 0 tasks (0%)
    
  • build 1809 with continuation after a stop on Stop, terminal did not restart, continuation.

    as the test is started from the beginning, from the proposed 10240 passes was made clean 5863 and 4377 were taken from the cache (4377 / 10240 = 42% cache hit)
    2018.04.30 11:04:28.046 Tester  Best result 3391.17 produced at generation 27. Next generation 38
    2018.04.30 11:04:28.477 Tester  genetic calculation is over
    2018.04.30 11:04:28.477 Tester  result cache used 4377 times
    2018.04.30 11:04:28.477 Tester  genetic optimization finished on pass 10240 (of 49644595)
    2018.04.30 11:04:28.488 Statistics      optimization done in 0 minutes 36 seconds
    2018.04.30 11:04:28.488 Statistics      local 5863 tasks (100%), remote 0 tasks (0%), cloud 0 tasks (0%)
    



Now the conclusions:

  1. Your statement about "3 times slower" is wrong.

  2. In fact, the tester of build 1809 has become faster through optimization and rewriting of internal processes.

    This can be seen in the time of full pass, it's 33 seconds instead of 38 seconds. And 33 seconds for a larger number of passes.

    If we recalculate number of passes per second, it turns out:
    - build 1755 gives 4780 passes / 38 sec = 125 passes per second
    - build 1809 gives 5549 passes / 33 sec = 168 passes per second

  3. The new scheme of "continuation after stop" genetic optimizer with generation of new population plan is correct.

    We are re-evaluating the tester concept and fixing old bad decisions.

    Since genetic tests need to be run multiple times to cover the search area more completely, our new model with visualisation of previous runs from the cache allows a better understanding of the mechanics of the genetics workflow.

    The new model for displaying optimization results with all the data from previously calculated caches added to the table allows to see the picture more completely. It is possible to compute your strategy in chunks and see full results every time, including previous runs.

    It is because of the really large merged results tables that we have disabled the rieltime display of the results list, and show the cumulative result after merging all previous results.
 

My robot crashes during initialisation, both in Runtime and in Debug. The compilation is fine.

There's a class with static members in the code, probably because of them. Everything was working fine before. But now it reports an error during global initialization.

Removed static members from the class, it runs fine.

 
Dennis Kirichenko:

My robot crashes during initialisation, both in Runtime and in Debug. The compilation is fine.

There's a class with static members in the code, probably because of them. Everything was working fine before. Now it writes an error during global initialization.

Removed static elements in class, launched normally.

Write your request to Service Desk and attach the Expert Advisor (can be an ex5 file), please.

 
Aleksey Vyazmikin:

All this is great!

However, please bring back the "Optimisation" tab - it's impossible to work without operational analytics! Large tables are slow - let's make a filter - show the top 20 for each criterion - it's not so resource-intensive(?), but it will help to see the picture very much. And, those billions of passes - who makes them? They are units with huge capacities - you yourself are talking about the reasonableness of applying genetics, and there are no such portmanteaus there. So more than 10k passes is a rarity.


Sad.

At least it was possible to build a picture on preliminary data while the optimisation was going on.

 
Renat Fatkhullin:

We are busy with a big performance upgrade of the tester and are redesigning the heavy load modes. Major improvements have been made and new acceleration methods will be implemented soon.

The aisle list window decided to show at the end of the miscalculation, so as not to waste real resources on maintaining, re-sorting and displaying the ever-changing aisle list.

There really was a huge waste of resources and slowdowns. Especially when we're talking hundreds of thousands of rows, millions and tens of millions of passes. There's no reasonable sense in looking at a bunch of preliminary data with your eyes.


We do our own optimization and run tests with 100 million full passes.

It's clear that with such numbers it's out of the question to re-sort in real time and display a table of 2-5-10-50 million values. There is only one option - to gather everything quickly and economically, to final sort it and provide a view of any depth.

Tell us about it, and that's it.

Reason: