Testing the new MQL5 compiler for x64 platforms - 2 to 10 times faster calculations! - page 20

New comment
 
Denis Kirichenko:

Optimise the logic. For example, work with arrays and loops. Try to pack the criteria values into an array. And do checks in a loop. Maybe there won't be a need in 74 thousand cases then...

Of course, in theory, you can generate each string by a long hash, and pass only these hashes, and then use it to generate everything - but I'm not sure that this will be fast and the task is not simple...

 
Alexey Kozitsyn:

1. you will see the most 'slow' places in the code there. Although... it's another question whether it affects the compilation...

2. As you like: you can use case. You were advised to break it into small functions. Break it up and test it. Yes, of course, the code will get larger. But what to do.

And here I have rewritten the code into functions - in the appendix.

I noticed at once that the previous code took 14428 kb after compilation and the new one took 9447 kb - I'm already surprised at the difference of 5 Mbytes - where from!

Further on by the compilation speed, the former

0 error(s), 0 warning(s), 2109302 msec elapsed

new version

0 error(s), 0 warning(s), 386131 msec elapsed

The new version is 5.46 times faster to compile.

And here is the previous version in terms of speed:

2019.10.15 14:35:47.593 Core 1  pass 0 returned result 1001000.000000 in 0:00:29.555
2019.10.15 14:35:47.595 Core 3  pass 4 returned result 1001000.000000 in 0:00:29.490
2019.10.15 14:35:47.605 Core 2  pass 2 returned result 1001000.000000 in 0:00:29.540
2019.10.15 14:35:47.641 Core 4  pass 6 returned result 1001000.000000 in 0:00:29.541
2019.10.15 14:36:15.511 Core 2  pass 3 returned result 1001000.000000 in 0:00:27.907
2019.10.15 14:36:15.523 Core 1  pass 1 returned result 1001000.000000 in 0:00:27.932
2019.10.15 14:36:15.535 Core 3  pass 5 returned result 1001000.000000 in 0:00:27.942
2019.10.15 14:36:15.537 Core 4  pass 7 returned result 1001000.000000 in 0:00:27.897
2019.10.15 14:36:15.537 Tester  optimization finished, total passes 8
2019.10.15 14:36:15.547 Statistics      optimization done in 0 minutes 58 seconds
2019.10.15 14:36:15.547 Statistics      shortest pass 0:00:27.897, longest pass 0:00:29.555, average pass 0:00:28.725
2019.10.15 14:36:15.547 Statistics      8000 frames (3.14 Mb total, 412 bytes per frame) received
2019.10.15 14:36:15.547 Statistics      local 8 tasks (100%), remote 0 tasks (0%), cloud 0 tasks (0%)

the new version.

2019.10.15 14:33:51.458 Core 3  pass 6 returned result 1001000.000000 in 0:01:01.840
2019.10.15 14:33:51.485 Core 2  pass 4 returned result 1001000.000000 in 0:01:01.867
2019.10.15 14:33:51.521 Core 1  pass 2 returned result 1001000.000000 in 0:01:01.903
2019.10.15 14:33:51.524 Core 4  pass 0 returned result 1001000.000000 in 0:01:01.906
2019.10.15 14:34:18.802 Core 3  pass 7 returned result 1001000.000000 in 0:00:27.346
2019.10.15 14:34:18.837 Core 2  pass 5 returned result 1001000.000000 in 0:00:27.354
2019.10.15 14:34:18.892 Core 4  pass 1 returned result 1001000.000000 in 0:00:27.370
2019.10.15 14:34:18.922 Core 1  pass 3 returned result 1001000.000000 in 0:00:27.403
2019.10.15 14:34:18.922 Tester  optimization finished, total passes 8
2019.10.15 14:34:18.932 Statistics      optimization done in 1 minutes 29 seconds
2019.10.15 14:34:18.932 Statistics      shortest pass 0:00:27.346, longest pass 0:01:01.906, average pass 0:00:44.623
2019.10.15 14:34:18.932 Statistics      8000 frames (3.14 Mb total, 412 bytes per frame) received
2019.10.15 14:34:18.932 Statistics      local 8 tasks (100%), remote 0 tasks (0%), cloud 0 tasks (0%)

And here we see that the first pass of agents (4 agents) is very slow - I tried it many times - the result is stable, but in the log

2019.10.15 14:38:07.002 Tester  OnTesterInit works too long...

What's it got to do with this now, maybe@Renat Fatkhullin or@Slava can tell me why such an effect occurs?

 
Andrey Khatimlianskii:

The file is zipped. Read the zip, unzip inside. It will be faster than transferring a 500 MB EA (it's also transferred to each agent).

Doesn't it then get decompressed again with every new pass?

And would reading from the file be faster than a one-time transfer of....

 
Aleksey Vyazmikin:

Doesn't it then get unpacked again each time a new pass is made?

Yes, and would reading from a file be faster than a single pass....

Yes, it might be slower with optimization... But I would check, everything is set up for that.

 
Andrey Khatimlianskii:

Yes, it might be slower during optimisation... But I would check, everything is ready for it.

What exactly is ready - I don't understand.

 
Aleksey Vyazmikin:

What exactly is ready - I don't understand.

Working with zip archives.

 
Andrey Khatimlianskii:

Working with zip archives.

Yes, I've seen it, but haven't tried it out in practice.

It's more of a problem for me in data preparation, i.e. translating the code into a table - I'll have to process the raw data again...

 
Unfortunately, it turned out that the compiler can't cope with large code - got the error "EX5 write error" - no other errors. It would be good to write about the limitations in the user manual!
 
Made a public version of the EA, now I'm checking - will it compile or not - the process is not fast, but now you can see that 46% of the code is compiled and already 36 GB of RAM is eaten...
 
Aleksey Vyazmikin:
Made a public version of the EA, now I'm checking - will it compile or not - the process is not fast, but now you can see that 46% of the code is compiled and already 36 GB of RAM is eaten...

Please provide me the code to investigate.
I'll check why it compiles so slowly and consumes so much memory.

1...13141516171819202122
New comment