All (not yet) about Strategy Tester, Optimization and Cloud - page 6

 

Forum on trading, automated trading systems and testing trading strategies

Myql agent manager

Alain Verleyen, 2017.09.18 19:27

Uncheck the "Active last day" on your mql5.com profile "Agents".

 

"The service did not respond to the start or control request in a timely fashion. (1053)" fixing in the latest builds:


Kenneth Parling:

as far as i can tell there's no were to find a solution for this error;

"EF 2 22: 34: 59.829 Service Service 'MetaTester-1' start failed [The service did not respond to the start or control request in a timely fashion. (1053)]"

I have manually tried to start my agents via Meta Tester agent manager and via my computers task services management without any success. Still 'stopped' and don't respond within time....

There is the reply -

Forum on trading, automated trading systems and testing trading strategies

New version of the MetaTrader 5 build 2170 platform: Scopes in MQL5, a global update of the strategy tester and integrated hosting

Sergey Dzyublik , 2019.10.10 09:02

The problem is most likely in the account under which the service starts. It used to be SYSTEM, in the latest versions of MT it is NETWORK SERVICE.
On Windows Server 2012 R2 there was the same problem.
It is solved by changing the account under which the service will be launched on the SYSTEM account:

1) Windows + R key combination, then enter services.msc
2) in the list of services you need to find the necessary services from MetaTrader 5 Strategy Tester Agent, their default names: MetaTester-1, MetaTester-2, MetaTester-3, ... MetaTester-N.
3) for each service, open the properties, go to the "Log On" tab, set "Log On as" as "Local System account" and click OK.



 

With this last build, the installation gives me only 4 agents on a new machine, and add button is grey, but my processor has 4 physical cores and 8 logical. So, I wanted add remotely the agents, but unlucky, it seems the command "C:\Program Files\MetaTrader 5 Strategy Tester\MetaTester64.exe /install address:0.0.0.0:30004 /password:MetaTester" add a agent with the wrong port 1024 (wathever the address and the port I give). Build 2170/2171 gave me the same error.

Sadly, I didn't kept a older version of the setup, is there a place to get the older install or it will be fix hopefully in the future? Or, have you restricted to physical cores/half of the logical cores, for performance reason ?

(running on Windows 10 Pro version 1903)

 
Florian Lhermite:

With this last build, the installation gives me only 4 agents on a new machine, and add button is grey, but my processor has 4 physical cores and 8 logical. So, I wanted add remotely the agents, but unlucky, it seems the command "C:\Program Files\MetaTrader 5 Strategy Tester\MetaTester64.exe /install address:0.0.0.0:30004 /password:MetaTester" add a agent with the wrong port 1024 (wathever the address and the port I give). Build 2170/2171 gave me the same error.

Sadly, I didn't kept a older version of the setup, is there a place to get the older install or it will be fix hopefully in the future? Or, have you restricted to physical cores/half of the logical cores, for performance reason ?

(running on Windows 10 Pro version 1903)

Okay, my bad, a slash was missing. /install /address  (not /install address). But, it still installs only 4 agents on a 8 logical cores CPU with the 2190 build, and button is grey, so its only possible to install with this command.

 
Florian Lhermite:

Okay, my bad, a slash was missing. /install /address  (not /install address). But, it still installs only 4 agents on a 8 logical cores CPU with the 2190 build, and button is grey, so its only possible to install with this command.

Open the Agents Manager, uninstall all the agents, then uninstall MetaTrader 5 completely, and reinstall all.  This should solve the problem.  I had something similar to this happen on a couple PCs a couple months ago when they did the big update to the Strategy Tester.  The tester uninstalled all but one agent, and I could only add half of them back until I followed the above procedure.
 
Over-optimization (curve fitting) - good discussion thread with some ideas and knowledge.
Over-optimization (curve fitting)
Over-optimization (curve fitting)
  • 2019.11.19
  • www.mql5.com
Hello Traders, How do you solve the problem of over optimization...
 

Forum on trading, automated trading systems and testing trading strategies

New version of MetaTrader 5 build 2190 platform

Renat Fatkhullin , 2019.11.22 11:08

I explained in detail with the repetition of one main idea: once a running agent lives up to 5 minutes in anticipation of any next task.

Agent doesn’t care - “first optimization” or “any single pass”. It sits and waits for the next task for 5 minutes, holding the initial data of previous calculations in order to quickly read the next order without painfully lifting the data from disk and unpacking it.

Since the agent was holding 2 GB, it means that it stored the raised data from previous single / any passes, as you indicated in your message. It automatically releases resources by a timeout of non-use.

 

Forum on trading, automated trading systems and testing trading strategies

Metatester 5 agent manager strategy tester does not see processor threads.

Renat Fatkhullin , 2019.11.27 06:12

Due to the apparent lack of memory with an excessive number of agents and a decrease in the speed of calculations on hyper-threading cores, we decided to limit ourselves to only physical cores when working in cloud.

Now in Cloud they drive very expensive tasks, and the vast majority of Cloud participants have a banal lack of memory with many agents. For example, 2 GB per agent and 16 agents already require at least 32 GB of memory, although usually 8-16 GB are available.

As a result, many passes fail, wasting the network and agents. But most importantly, they are slowing down the calculation due to expectations of results and restarting tasks.

We have long been evaluating the approximate resource sufficiency of agents before issuing tasks to them, and one of the most effective is to work only on physical cores in cloud.

Locally, you can use all the cores as you can easily control their shutdown.


Forum on trading, automated trading systems and testing trading strategies

Metatester 5 agent manager strategy tester does not see processor threads.

Renat Fatkhullin , 2019.11.27 07:18

After the release, we undertake a radical restructuring of the resource management of the tester and its agents.

This primarily affects the effectiveness of local agents.


 
New MetaTrader 5 Platform beta build 2245: DirectX functions for 3D visualization in MQL5 and symbol settings in Strategy Tester
New MetaTrader 5 Platform beta build 2245: DirectX functions for 3D visualization in MQL5 and symbol settings in Strategy Tester
  • 2019.11.29
  • www.mql5.com
The beta version of the updated MetaTrader 5 platform will be released on November 29, 2019...
 

Good article was published -

----------------

Continuous Walk-Forward Optimization (Part 1): Working with Optimization Reports 

In the previous articles (Optimization Management (Part I) and Optimization Management (Part 2)) we considered a mechanism for launching the optimization in the terminal through a third-party process. This allows creating a certain Optimization Manager which can implement the process similarly to a trading algorithm implementing a specific trading process, i.e. in a fully automated mode without user interference. The idea is to create an algorithm which manages the sliding optimization process, in which forward and historical periods are shifted by a preset interval and overlap each other.

This approach to algorithm optimization can serve as strategy robustness testing rather than pure optimization, although it performs both roles. As a result, we can find out whether a trading system is stable and can determine optimal combinations of indicators for the system. Since the described process can involve different robot coefficient filtration and optimal combination selection methods, which we need to check in each of the time intervals (which can be multiple) the process can hardly be implemented manually. Moreover, thus we can encounter errors connected with data transfer or other errors related to the human factor. Therefore, some tools are needed that would manage the optimization process from the outside without our intervention. The created program meets the set goals. For a more structured presentation, the program creation process has been split into several articles, each of which covers a specific area of the program creation process.

This part is devoted to the creation of a toolkit for working with optimization reports, for importing them from the terminal, as well as for filtering and sorting the obtained data. To provide a better presentation structure, we will use the *xml file format. Data from the file can be read by both humans and programs. Moreover, data can be grouped in blocks inside the file and thus the required information can be accessed faster and easier.

Our program is a third-party process written in C# and it needs to create and read created *xml documents similarly to MQL5 programs. Therefore, the report creation block will be implemented as a DLL which can be used both in MQL5 and C# code. Thus, in order to develop an MQL5 code, we will need a library. We will first describe the library creation process, while the next article will provide description of the MQL5 code that works with the created library and generates optimization parameters. We will consider these parameters in the current article.

Continuous Walk-Forward Optimization (Part 1): Working with Optimization Reports
Continuous Walk-Forward Optimization (Part 1): Working with Optimization Reports
  • www.mql5.com
In the previous articles (Optimization Management (Part I) and Optimization Management (Part 2)) we considered a mechanism for launching the optimization in the terminal through a third-party process. This allows creating a certain Optimization Manager which can implement the process similarly to a trading algorithm implementing a specific...
Reason: