MetaTrader 5 Strategy Tester and MQL5 Cloud Network - page 2

 
Interesting:

Here's where it gets more detailed. As far as I understand it, there is one core per run, but you can choose which one + it is possible to run several terminals.

For single run (not in optimization mode, but single run) you can choose one of the local agents or one of the remote ones (working in server mode).

For a single run, you can't select an agent from the MQL5 Cloud Network, because it makes no practical or economic sense.

Roughly speaking, it makes no sense to initialize a powerful distributed MQL5 Network system with cache raising and data preparation for a single test. The purpose of the MQL5 Cloud Network is to perform massive optimization calculations.

 
Renat:

For a single run (not in optimization mode, just a single pass), you can select one of the local or one of the remote agents (working in server mode).

For a single run, you can't select an agent from MQL5 Cloud Network, because it makes no practical or economic sense.

Roughly speaking, it makes no sense to initialize a powerful distributed MQL5 Network system with cache raising and data preparation for a single test. The purpose of the MQL5 Cloud Network is to perform massive optimization calculations.

By the way, I've thought about it - can we make some kind of preliminary "calculation", e.g. analyzing cache transfer speed (rise time) as a minus and pluses of more cores and comparing it with only local testing time to give some "advice" - there is no sense to run such calculations on remote ones. In fact it is not obvious at all at first.
 
Renat:

For a single run (not in optimization mode, just a single pass), you can select one of the local or one of the remote agents (working in server mode).

For a single run, you can't select an agent from MQL5 Cloud Network, because it makes no practical and economic sense.

Roughly speaking, it makes no sense to initialize a powerful distributed MQL5 Network system with cache raising and data preparation for a single test. The purpose of the MQL5 Cloud Network is to perform massive optimization calculations.

So I've got it right, one local/remote agent + several terminals
 
Academic:
By the way that thought - can be in terms of "improvement" to make such kind of preliminary "calculation", say, having analyzed speed of transfer (time of rise) of caches as minus and those pluses from more cores and to compare received with only local testing time to give some "advice" - like there is no sense to run on remote ones. In fact it is not obvious at all at first.

Of course, the test manager will try to use more local agents, then remote agents, and then distributed agents to reduce the cost of computation.

In addition, we are introducing a batch processing mechanism, which can reduce the impact of network delays when working with remote agents to almost zero.

That is, the terminal will distribute tasks in blocks of 32-64 runs to each agent, which will reduce the impact of network delays by the same number of times.

Example: I sent a batch of 32 tasks in 2KB with parameters to calculate, and received a 1KB reply packet with results from an agent 5 minutes later. Bottom line is 3kb network traffic with about 1 second transmission loss instead of 32 seconds without packeting.

 

Thanks for the answers, but a lot is still unclear.

Для одиночного прогона (не в режиме оптимизации, а именно одиночный проход) можно выбрать одного из локальных агентов или одного из удаленных (работающих в серверном режиме).

What does it mean: "remote, running in server mode"? I just don't understand - if you install an agent on a second computer using the Metatester component, is this it? What about remote ones not in server mode - how do I add them?

For a single run, you can't select the agent from MQL5 Cloud Network, because it makes no practical or economic sense.

This is where we need a supercomputer, or rather a cluster of supercomputers working as a single core-as a single agent, and a network is required - no one has such a computer at home. Or at least the ability to connect to a powerful machine (it's possible, as I understand it, if I install the agent on a powerful computer, and use it from a laptop during a single run). Exactly for a single run. In fact, it turns out to be the opposite - there is no practical sense in using the MQL5 Cloud Network for massive optimization calculations, if even the initial single run is difficult. The enumeration of variants is the second case, but the single run is not less important, and even more important for some people.

So I understand correctly, one local/remote agent + multiple terminals

This one is not clear at all how to decipher it...

 

Renat:

Example: sent a task packet of 32 runs of 2kb with parameters to calculate and received 5 minutes later a reply packet of 1kb with results from the agent. The result is 3kb of network traffic with about 1 second of transmission loss instead of 32 seconds without packeting.

This is true if the history is primed and there is no additional overhead. But in principle traffic reduction and optimization efficiency improvement will be in your face.
 

Renat:

Roughly speaking, it makes no sense to initialize a powerful distributed MQL5 Network system with cache elevation and data preparation for a single test. The purpose of the MQL5 Cloud Network is to perform massive optimization calculations.

I agree with that. However, it is a pity that there is no possibility to run multiple single runs (test runs) simultaneously, even with multiple cores on a single machine. More specifically, sequentially, of course, but without waiting for the completion of previous runs. Several versions of the terminal are very expensive in memory. But if the tester was a standalone program, with the ability to run multiple instances, it would be better. Ideally - a multitasking tester. Now we have to get twisted - to write a configuration file with parameter lists and load them from a file by feeding the optimizer with a pseudo-variable-counter. It's not convenient. Not to mention the fact that the full set of test results (transactions, in particular) in this variant also has to be calculated, formatted and dumped into a file in the deinit. So, mass processing of optimization results is quite complicated in the current version of the tester. You cannot even upload yesterday's testing results from a file (which does exist!) back to the "optimization results" page to solve them having a "one-click" tester at hand. Can you at least implement such a possibility?

One more sore point: I understand that during optimization it takes quite a long time to prepare data for agents. Is it possible to load agents not with single runs-tasks, but with batch runs (8-16-32)? In this case (imho) we could get a tangible gain in total optimization time. As far as I know such a scheme is successfully used in F4 now. I think several sets of parameters are run in parallel (I could be wrong). So, I'd like to have something like that on firth. Otherwise my single-core tester lags many times behind the four-core one (I've already written about it once).

//Wow! Too late. By the time I wrote, Renat already wrote about batch processing in the affirmative. Thank you. Very pleased. We will wait.

 
-Alexey-:

Thanks for the answers, but much remains unclear.

Which means: "remote, running in server mode"? I just don't understand - if you install an agent on a second computer using the Metatester component - is this it? And what about remote, non-server mode - how do I add them?

This is where we need a supercomputer, or rather a cluster of supercomputers, working as one core as one agent, and a network - no one has one at home. Or at least the ability to connect to a powerful machine (it's possible, as I understand it, if I install the agent on a powerful computer, and use it from a laptop during a single run). Exactly for a single run. In fact, it turns out to be the opposite - there is no practical sense in using the MQL5 Cloud Network for massive optimization calculations, if even the initial single run is difficult. Trying out variants is the second case, but single single run is not less important, and even more important for some people.

It's not at all clear how to decipher this...

1. A single run uses only one core, local (your PC) or remote agent (intranet).

2) Certain cores can be disabled.

3. you can select a specific agent (a specific core) on which to test.

It is theoretically possible to perform several "single tests" at the same time (but then multiple terminals would be required).

PS

In your case with laptop you should switch off local kernels and do the test on a powerful computer (which is in local network or which resources will be maximal free for tests).

 

MetaDriver:

Is it possible to load agents not by single runs-tasks, but by packages of runs (8-16-32)? In this case (imho) you can get a tangible gain in total optimization time. As far as I know such a scheme is successfully used in F4 now.


So they implement batch mode, Renat even gave an example...
 

What I don't understand is....

  1. What about the history? Will it be the same for all? What if I have downloaded a terminal from different brokerage companies with very little history and moreover it has holes in different places?
  2. If the number of instruments is not the same, the example on the server is 12 symbols of the championship. And for testing (multicurrency needs a full currency matrix for the indicator to work properly) how in this case ? ....
  3. And thirdly, we already mentioned time, that's why we introduced time UTG - to synchronize somehow ... How will it be like with you? If, say, only certain trading hours are tested (for example from 10 to 12 in Moscow) ... The time is different for everyone ...
Reason: