Testing the new MQL5 compiler for x64 platforms - 2 to 10 times faster calculations! - page 14
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
I wanted to test acceleration of calculations on my Expert Advisor today in anticipation of super speed (I even used a faster CPU) and... For some reason the optimized Expert Advisor was running in the tester three times slower. Has it never happened to you? As soon as I localize and reproduce the problem, I will post the code. There are no system calls there, only operations with a bunch of arrays and basic arithmetic operations.
Also, here are a couple of requests:
1. We may add the possibility to launch the script from the editor without debugging, i.e. as if it was done from the Navigator. This kind of launching is present in any development environment. I was thinking to suggest Ctrl + F5, as in sys, but the combination is already taken by you :)
2. Add invoking wind's console stdout from script, to avoid switching to terminal every time to see the script's result. Although, as I understand it, you can try to call it using WinAPI.
3. Add at least some genetic optimization settings. For example, number of combinations can be 1e35 and 10k passes is not always enough (optimizer doesn't even generate new passes after 10k), you understand that 20-30k would be enough, but you can't tune it. On the other hand, when you fix some parameters and leave some for more fine tuning and studying distribution... So, when the number of combinations is small, for example, 50k, the optimizer for GA offers only 2k runs - and this is also not enough for this particular task - to better study behavior of the function on these parameters. I.e. it is stupid to go through all 50k, while GA generates only 2k runs and does not give a sufficient picture. I hope you understand what I'm talking about.
Improvements are sure to come.
Has anyone already tested the debugging on history in the strategy tester?
Improvements are sure to come.
Has anyone already tested the debugging on history in the strategy tester?
Improvements are sure to come.
Has anyone already tested debugging on history in the strategy tester?
It would also be nice to be able to select "find usages" from the context menu.
I know it's a bit off-topic, but I don't need to write any changes, but it's nice for developers to have such things.
I have checked Moving Average, MACD Sample on the new compiler, but I have not found any differences. I will try it on my EAs.
I haven't fully debugged and found the bug (very differently), but so far I've found the following problems with Optimize=1. Let me tell you right away that my Expert Advisor has about 100 input parameters, and it works a lot with arrays. In addition, it does not trade, it only performs calculations.
I will describe the order of my actions:
1. I launch the terminal, select "Optimization = Disabled" in the tester and give a master set of input parameters. The tester gives the correct result.
2. I choose "Optimization = Fast (genetic algorithm)", launch it - the tester doesn't even generate the first step. Agents (processors) don't respond. At first I thought there may be some calculation errors with bad parameter sets and the tester just dies.
3. Again I select "Optimization = Disabled" and again give the reference set. Now the tester does not show any results at all.
4. I open EA code in the editor, press F5 to generate a non-optimized code. In the tester, I again run it on the reference set - voila, there is a result. Well, something is glitchy inside the Expert Advisor due to optimization.
I have added a function that logs the Expert Advisor input parameters into a file. So, at step 2 (optimization is running and the tester's steps are almost not generated), random values were popping up for some parameters which are not provided at all because the parameter is either unchangeable or the value is outside the set range. For example, I had one parameter set to 80 and it was invariant. In the log it was set to 0.01355835795402527 (?). The other was set to 0 and logged as 4.940656458412465e-324; Both parameters were double. But then I found out that there were totally inappropriate values for int, too.
Second, I cut out the "unnecessary" code of the Expert Advisor to leave only that code which also causes an error. Still it turned out to be rather much. Plus the logging function was outputting the value of ALL parameters. However, when I removed most of parameters (and they became less than 64 or 80), the tester didn't die. That said, I also found a strange connection between the error and my logging function. When it's commented out (i.e. all parameters are NOT used in the code), the tester dies. When it logs, the tester dies. Very strange.
And third. I noticed that paralleling calculations does not result in a linear increase in overall speed of runs. For example, I am attaching the files: a script and an Expert Advisor with the same code. This is a real piece of code, cut out from my Expert Advisor, but the arrays are renamed. When Optimize=1, my script runs in 156 ms, i.e. maybe 6 runs per second. (Without Optimize it takes 3 times longer). I have 8 threads, i.e. theoretically 48 runs per second are possible. When running the Expert Advisor in the tester (do Slow Optimize, enable the dummy parameter, for example, for 2000 runs) I only got a little over 4 runs per second, i.e. even slower than on a single core! Although there were honestly 8 processes hanging in the task manager. Why is this so? Multiple processes blocking access to memory? Long initialization of arrays? (Admittedly, arrays are not big there).
When we run a single test in the tester, Print: 156 ms. That is, everything seems to be normal with the Expert Advisor.
By the way, will there ever be pointers in MQL? :) Because I had to make arrays-copies.
For some reason, expert with optimization worked 3 times slower in the tester.
Now I understand why it's getting slower - only one agent (random) works, the others fail and so constantly one attempt every few seconds. Nothing is clear from the tester log. As I understand it, if the most recent agent fails as well, then what I wrote about earlier happens: the tester steps are not generated at all.
Build 1117 did not solve all the problems described above (including struct array).
Updated to 1117 today, but debugging on historical data - still not going.
I take ExpertMACD.mt5, put a breakpoint right at the entry to the Init function, start debugging on historical data (press the button), the editor window closes, strategy tester settings open. I set the time period (for 2015, diaries, all ticks), visualization - the checkbox is on, I press the Start button and the visualization starts. Breakpoint did not work.
What am I doing wrong ?
Updated to 1117 today, but debugging on historical data - still not going on.
I take ExpertMACD.mt5, put a breakpoint right at the entry to the Init function, start debugging on historical data (press the button), the editor window closes, strategy tester settings open. I set the time period (for 2015, diaries, all ticks), visualization - the checkbox is on, I press the Start button and the visualization starts. Breakpoint did not work.
What am I doing wrong ?
I need to open a demo account onMetaQuotes-Demo, then
My MACD Sample.mq5 on a moex demo account the USDRUB_TOM symbol does not work (first it starts and then stops after a few steps) and a simple backtest does not work either.
But Moving Average.mq5 on a demo account moex USDRUB_TOM works fine (and so does the simple backtest).
On the demoforex demo account the EURUSD symbol works fine (and so does the simple backtest) MACD Sample.mq5 and Moving Average.mq5
Recorded a debugging video on MACD Sample.mq5 EURUSD symbol on demoforex account. (daily works fine too)
I haven't fully debugged it and haven't found how the bug comes out (in very different ways), but so far I've found the following problems when Optimize=1. Let me tell you right away that my Expert Advisor has about 100 input parameters, and it works a lot with arrays. Plus it does not trade but only performs calculations.
...............................................
By the way, will there ever be pointers in MQL? :) Because I've had to make arrays-copies.
Hello, I would like to be more specific.
1. Your Expert Advisor has made calculations, but has not traded. What and how does the Expert Advisor display? What do you expect from the Expert Advisor when running it in the strategy tester? What data do you expect to get during testing with optimization? We have not found any errors during testing of the attached Expert Advisor. Do the errors you describe relate to it?
2. step by step what you do and what you expect, please attach the code you use.
3. can you attach the testing logs with and without optimization?
4. Your terminal version, OS?
Eduard Vavrin, the attached file is not related to the described errors. Let me try to prepare the code that leads to the error and send it to you. It just takes time and so far this code contains too much from the Expert Advisor's source.
Regarding the file I attached: it was saying that a single run of the attached EA takes 156 ms, i.e. 1/6 of a second. We can assume that on 8 threads we would get 48 runs per second. In reality, the multi-runner (there is the dummy parameter, Slow Optimization mode) yields only 4-5 runs per second on 8 threads. The question was: why is the speed dropping? On the Agents tab I can see that all threads are running and each thread spends more than 1 second for a run. What the Expert Advisor returns rand() is just a tick to see that it worked.