MetaTrader 5 Strategy Tester and MQL5 Cloud Network - page 16

 
Interesting:

Is it such a difficult task to make a direct agent manager for Linux (at least for the most popular ones)?

The thing is that such a simple file like metatester[64].exe (~2.5 Mb) includes

  1. the testing agent itself, including the entire runtime subsystem of MQL5, its sandbox and the entire business logic of the terminal
  2. All GUI part of the testing visualization (this is a pure Windows GUI without a chance to be ported)
  3. GUI part of the agent manager
  4. MQL5 Cloud Network
  5. auto updates

We have packed into one EXE file, which does not require any other additional libraries/components, so many functions and simplified the work with it, that the replication of the same functionality with the same quality under Linux is out of the question.
 
Renat:
The fact that we packed into one EXE file, which doesn't require any other additional libraries/components, so many functions and simplified the work with it, that the replication of the same functionality with the same quality under Linux is out of the question.

Microsoft's perennial cross-platform problem.

Renat:

The thing is that such a simple file as metatester[64].exe (~2.5 Mb) includes

  1. the testing agent itself, including the entire runtime subsystem of MQL5, its sandbox and the entire business logic of the terminal
  2. All GUI part of the testing visualization (this is a pure Windows GUI without a chance to be ported)
  3. GUI part of the agent manager
  4. MQL5 Cloud Network
  5. auto-updates

It is clear. I don't understand why they can't cut some of the functionality and leave only what is available in MQL5 Cloud Network?

For example, I don't understand the need for such a scheme of the second item (the main point of which is the impossibility of porting)...

 
Interesting:

Microsoft's perennial problem with cross-platforming.

This is understandable. I don't understand why they can't cut some of the functionality and leave only what works in MQL5 Cloud Network?

For example, I don't understand the need for such a scheme of the second item (the impossibility of porting is the main point)...

I'd rather ask myself more important questions:

  1. Who needs a stripped-down tester for Linux? Without protections, without normal 32/64 support, without nice architecture and simplicity. Developers definitely don't need it.
  2. Why will the developers spend half a year (minimum) of their lives on this project spitting and cursing? Where is the developer's profit?
  3. Why will the developers ruin a clean architecture by introducing a cross-platform horror into the code?

Software development is a business driven by clear answers to economic questions.

Those who don't take hard economics into account go and do air projects, bankrupt investors, and then... change jobs and usually again in a circle :)

ps: MetaTrader 4 and MetaTrader 5 work quite decently under Linux via Wine, and we explicitly make sure that the terminal works well in Wine (we fix the little things from time to time).

 
Renat:

In the next version, tester agents will not be able to work in the MQL5 Cloud Network, if they are launched under virtualizers or emulators (Virtual PC, VMWare, Virtual Box, Wine, etc.). This is done to minimize the possibility of inflated performance ratings.

The normal server mode of running agents under virtualisers is fully preserved. This means that you can use agents in any form for your own needs.

what is the problem, what kind of overclocking is taking place under virtualization?

what does it matter where the kernels are?

really sorry if it's possible to spin the rating , honestly i dont really understand how ...

what is rating? what is rating?

there is a task for the kernel - the kernel completes the task - there is a time for the kernel to complete this task

there should be no other criteria

all this can be controlled in the MQL5 Cloud Network

Simply put

1-core says I'm ready

2- MQL5 Cloud Network says to the core, go to work, set the time

3-core works for 3 seconds

4-core responds to MQL5 Cloud Network -> take your task

5 - well done kernel - you did it in 3 seconds - here is candy in the form of 0.1 cents

another core

1-Core said I'm ready.

2- MQL5 Cloud Network - > kernel is on time

3-core works for 30 seconds

4-core answers to MQL5 Cloud Network -> take your task

5 - well done core - you did it in 30 seconds - here is candy in the form of 0.1 cents

it's simple - the faster and the harder you work, the more you get.

The MQL5 Cloud Network should not have favorites - the tasks will be given to them more often ...

It's like in life - favorites get orders more often!)

meaning that the Cloud Network - should not just distribute tasks by list ! but probably by moving the cores to the queues

to prevent the situation when only those at the top of the list will get the job - and so on in a circle

if we assume there are 100 tasks and about 1000 cores

then the first 100 will get the job

they will complete them - then another 100 tasks will be queued up and again they will be handed over to the cores which are at the front of the list

and the bridge 900 will be idle

so the first 100 cores will be the most likely to earn

---

well if the problem is not solved then alas

:-(((( , then minus 8 of my good cores from the pool, I7 - 3 GHz, 8 gigs of cores

i won't be leaving the virtual machine - it's under LINUX CentOS , i won't be moving to windows host .

this is exactly the host that works 24/7!

----

i purposely just did it on a virtual machine and allocated 2 cores out of the 3 cores on the host to the virtual machine

I now have 47 cores in the pool - of which only these last two

are on the same machine

64x 3 amd cores

i.e. of the 3 physical cores two are being shared a second time but from a different operating system

Renat, I don't quite understand why this would be a problem

If it helps to test and find a solution - allowing the use of virtual machines will be glad

I can make a couple more of these if needed

By the way - out of 47 cores only these two have been distributed 2 times

the others are all physically one time - even the ones on a large i7 machine are all on virtual machines but only one time

--

 
The problem is that you can't charge "per task". Payment goes for resources used = CPU performance (PR) * time.

In this formula, an accurate estimate of performance is critical, which is easy to fudge in virtualisations.
 

On one of the computers I have installed and run test agents, but they don't work. More precisely, the CPU time is periodically occupied by metatester services (up to 100% per core), but the agents manager shows zeros (tasks and time; local and cloud) and on mql5 site these agents have zeros in profile (but agents are visible in profile, during one of the runs somehow one of them had one task and then another one). When accessing these agents from MetaTrader5, you get messages that they are busy.

Here is the content of the MT5tester directory (from which I run metatester.exe) and attached subdirectories and files:

MT5tester
    Tester
        Manager
            logs
                20110723.log
                20110725.log
            common.ini
    metatester.exe

I attach these files. As you can see, the Tester directory lacks the Agents* directories that are normally created.

Version metatester build 479, Windows Server 2003 R2 with SP2, Enterprise Edition, English, 32 bit; Intel Core 2 Duo T9300 2.5 Ghz CPU; 4 GB RAM. The MT5tester directory and all subdirectories and files for all users (Everyone) is set to Full Control, prior to this it is set to Full Control for the user who is running metatester.exe (and logging into Windows) and for the SYSTEM user who is running the metatester services.

Files:
pack.zip  2 kb
 

The agents were registered in MQL5 Cloud Network and were working on tasks from the network. At this point, the agents could not be connected as remote ones, as they were really busy.

Here are their incomplete (not all fields are updated yet) statistics - about 63 000 tasks have been performed:

Intel Core2 Duo T8300 @ 2.40GHz, 3581MB


0 32483 0 2011.06.08 2011.07.25
Intel Core2 Duo T8300 @ 2.40GHz, 3581MB


0 29861 0 2011.06.08 2011.07.25


The Agents directories were automatically removed when uninstalling these agents (they clean up their own rubbish). Here are the lines from the agent deletion log:

RQ    0    Tester    11:12:33    MetaTester 5 build 479 (12 Jul 2011)
HQ    0    Startup    11:12:33    initialization finished
NH    0    Service    11:12:47    Service 'MetaTester-1' stop initialized
MP    0    Service    11:12:50    Service 'MetaTester-1' stopped in 3031 ms
NH    0    Service    11:12:50    Service 'MetaTester-1' successfully uninstalled
PS    0    Service    11:12:51    Service 'MetaTester-2' stop initialized
KK    0    Service    11:12:56    Service 'MetaTester-2' stopped in 5500 ms
OS    0    Service    11:12:56    Service 'MetaTester-2' successfully uninstalled


In general, everything works as intended.

 
Renat:

The agents were registered in MQL5 Cloud Network and were working on tasks from the network. At this point, the agents could not be connected as remote, as they were really busy.

...

The Agents directories were automatically deleted during deinstallation of these agents (they clean up their own rubbish). Here are the lines from the log about deleting agents:

...

Meaning other agents:

CPU PR Tasks Revenue Created Activity
Intel Core2 Duo T9300 @ 2.50GHz, 4094MB 0 0 0 2011.06.08 2011.06.08
Intel Core2 Duo T9300 @ 2.50GHz, 4094MB 0 2 0 2011.06.08 2011.07.22

The Agents directories never showed up while the metatester services were running (I know about deleting data when uninstalling).

 
zdd:

Meaning other agents:

CPU PR Tasks Revenue Created Activity
Intel Core2 Duo T9300 @ 2.50GHz, 4094MB 0 0 0 2011.06.08 2011.06.08
Intel Core2 Duo T9300 @ 2.50GHz, 4094MB 0 2 0 2011.06.08 2011.07.22

The Agents directories never appeared while the metatester services were running (I'm aware of the fact that uninstalling deletes data).

These are the wrong agents - look at the dates of creation and last activity.

The correct ones to look at are the agents I cited - their last activity dates are exactly the same as their current date and their uninstall logs.

ps: install the agents again and look at the directories and their personal logs

 
Renat:

These are the wrong agents - look at the dates of creation and last activity.

It's correct to look at the agents I cited - their last activity dates are exactly the same as their current date and their uninstall logs.

ps: install the agents again and look at the directories and their personal logs

The agents you cited are running on another computer (with a T8300 processor), no issues with them.

Checking again (on computer with T9300 processor).

1. Uninstalled the agents. Uninstalled MT5tester directory with all its contents.

2. Uninstalled all the agents from the profile, which were on the computer with the T9300.

3. I created the MT5proba folder and gave it full access for Everyone.

I copied the file metatester.exe (build 479) into it (was obtained by LiveUpdate in MetaTrader5). 5.

5. ran metatester.exe, specified account, created two agents on new ports (one of them did not start for unknown reason, so created two more, deleted unnecessary ones). I am attaching the log file.

6. Started MetaTrader5 on another computer, deleted all Remote agents in strategy tester, created two new agents (specified IP-address, port, password). Selected each of them as a current agent and ran the single testing of an EA. The CPU type was detected, but the test was not executed.

MetaTrader5 Strategy Tester

I show the log:

CS      0       zddbase 15:38:19        connecting to 192.168.0.33:4001
CD      0       zddbase 15:38:19        connected
EM      0       zddbase 15:38:19        authorized (agent build 479)
II      0       Tester  15:38:19        EURUSD,M1 (MetaQuotes-Demo): testing of Experts\proba.ex5 from 2011.07.10 00:00 to 2011.07.16 00:00 to be started
NF      0       zddbase 15:38:19        common synchronization completed
HL      0       zddbase 15:38:19        3768 bytes of group info loaded
DM      0       zddbase 15:38:19        1478 bytes of tester parameters loaded
MJ      0       zddbase 15:38:19        3256 bytes of input parameters loaded
FN      0       zddbase 15:38:19        initial deposit 10000.00 USD, leverage 1:100
CE      0       zddbase 15:38:19        successfully initialized
MO      0       zddbase 15:38:19        13 Kb of total initialization data received
QF      0       zddbase 15:38:19        Intel Core2 Duo  T9300 @ 2.50 GHz, 4094 MB
FQ      2       zddbase 15:38:19        symbol EURUSD synchronization error
QD      0       zddbase 15:38:19        cannot get history EURUSD,M1
HO      0       zddbase 15:38:20        connection closed
CD      0       zddbase 15:38:34        connecting to 192.168.0.33:4002
MF      0       zddbase 15:38:35        connected
CL      0       zddbase 15:38:35        authorized (agent build 479)
GK      0       Tester  15:38:35        EURUSD,M1 (MetaQuotes-Demo): testing of Experts\proba.ex5 from 2011.07.10 00:00 to 2011.07.16 00:00 to be started
LD      0       zddbase 15:38:35        common synchronization completed
JN      0       zddbase 15:38:35        3768 bytes of group info loaded
NK      0       zddbase 15:38:35        1478 bytes of tester parameters loaded
KM      0       zddbase 15:38:35        3256 bytes of input parameters loaded
LH      0       zddbase 15:38:35        initial deposit 10000.00 USD, leverage 1:100
MS      0       zddbase 15:38:35        successfully initialized
CM      0       zddbase 15:38:35        13 Kb of total initialization data received
OD      0       zddbase 15:38:35        Intel Core2 Duo  T9300 @ 2.50 GHz, 4094 MB
LS      2       zddbase 15:38:35        symbol EURUSD synchronization error
OG      0       zddbase 15:38:35        cannot get history EURUSD,M1
PQ      0       zddbase 15:38:35        connection closed

During optimization:

EF      1       Tester  15:43:50        genetic mode turned on due to 4443670 passes, to use complete algorithm decrease amount of optimized parameters
LJ      0       Tester  15:43:50        Experts\proba.ex5 on EURUSD,M1 from 2011.07.10 00:00 to 2011.07.16 00:00
NL      0       Tester  15:43:50        genetic optimization started
DK      0       Tester  15:43:50        size of initial task pack is 128
JP      0       zddbase 15:43:50        connecting to 192.168.0.33:4001
IG      0       zddbase 15:43:50        connecting to 192.168.0.33:4002
PS      0       zddbase 15:43:50        connected
NO      0       zddbase 15:43:50        connected
HE      0       zddbase 15:43:50        authorized (agent build 479)
PO      0       zddbase 15:43:50        authorized (agent build 479)
NP      0       zddbase 15:43:50        genetic pass (0, 2628127, 1) started
HI      0       zddbase 15:43:50        genetic pass (1, 2957422, 1) started
GQ      0       zddbase 15:43:50        common synchronization completed
GJ      0       zddbase 15:43:50        common synchronization completed
NN      2       zddbase 15:43:50        genetic pass (0, 2628127) tested with error "no history"
KF      0       Tester  15:43:50        genetic pass (0, 2628127) returned to queue
QR      0       zddbase 15:43:50        connection closed
PG      0       zddbase 15:43:50        connecting to 192.168.0.33:4001
HS      0       zddbase 15:43:50        connected
RQ      0       zddbase 15:43:50        authorized (agent build 479)
LJ      0       zddbase 15:43:50        genetic pass (0, 2628127, 1) started
QS      0       zddbase 15:43:50        common synchronization completed
JI      2       zddbase 15:43:50        genetic pass (1, 2957422) tested with error "no history"
GM      0       Tester  15:43:50        genetic pass (1, 2957422) returned to queue
GJ      0       zddbase 15:43:50        connection closed

The logs show that the agent failed to load history. Nothing has changed in the agent directory tree compared to what I gave earlier (no Agents* directories, no history).

Agents have not appeared in mql5 profile yet, they are not loading CPU yet (it's been 1 hour since they started)...

Reason: