Design question about many symbol EA

 
1. If a many symbol EA is run [on any chart] and begins applying system on many symbols - we get many orders... duh (bear with me :-)
Since the EA works 24/7 their will be periods when the 'attached to chart' does not get good frequency of data tics. IE, the EA will be unable to effectively manage [in a timely manner] the open trades for other symbols.
So that is my design question - many trades could be open on many symbols and the attached chart which EA runs on may not receive data ticks regularly, meaning the other symbols with trades cannot be managed effectively (eg: trailing S/L either shadow or via ordermodify)
What do you suggest or do about it?
(easy answer is have eg, EA attached to 10 charts,... but that is not what I'm after)
.
2. In the middle of the night [sods law/typically] when server connection lost and EA sees "!IsConnected()".
Do you just hang about [like if EA not enabled], waiting for reconnect?
If so and orders at market, do you close all on reconnect or determine case by case if should close.
.
Am thinking (1) is most important...
.
Thanx in advance
 
fbj:
1. If a many symbol EA is run [on any chart] and begins applying system on many symbols - we get many orders... duh (bear with me :-)
Since the EA works 24/7 their will be periods when the 'attached to chart' does not get good frequency of data tics. IE, the EA will be unable to effectively manage [in a timely manner] the open trades for other symbols.
So that is my design question - many trades could be open on many symbols and the attached chart which EA runs on may not receive data ticks regularly, meaning the other symbols with trades cannot be managed effectively (eg: trailing S/L either shadow or via ordermodify)
What do you suggest or do about it?
(easy answer is have eg, EA attached to 10 charts,... but that is not what I'm after)
Personally, my EA's are designed to work on 1 pair only and to work in a multi-EA environment, so indeed I attach each EA to a different chart to trade on more than 1 pair. But u want a different solution... I can think of two ways to do it:
  1. Design the EA to run in a loop (every 100ms or so). This has many advantages but also many disadvantages. This kind of EA is discussed in the book in the chapter about complex programs -> https://book.mql4.com/special/index. One of the big disadvantages of running in a loop is the inability to access the properties window, but this could be overcome with the method discussed in the article "Changing the External Parameters of MQL4 Programs without Restarting" -> https://www.mql5.com/en/articles/1538 (this is not trivial though...). You also can't use this kind of EA in the Tester, so you would have to maintain another version that does not work in a loop for testing...
  2. Have a minimalistic dummy EA that uses the methods discussed here -> https://www.mql5.com/en/forum/119781 to send dummy 'ticks' from one chart to another chart. So the idea would be to have one of these EA's attached to each chart (each pair that will be traded) and have the main EA attached to one chart only. On each incoming tick on one of these charts, the dummy EA's would send a dummy 'tick' to the main EA's chart which would make it run. So we still need an EA per chart, but only one instance of the main EA is needed, the rest do almost nothing. To conserve memory, all other charts can be opened on a monthly timeframe.

2. In the middle of the night [sods law/typically] when server connection lost and EA sees "!IsConnected()".
Do you just hang about [like if EA not enabled], waiting for reconnect?
If so and orders at market, do you close all on reconnect or determine case by case if should close.
This highly depends on the EA's design and the algorithm implemented...
 
It just occurred to me that combining idea 1 and 2 would be perfect -> have one dummy EA that runs in a loop and checks changes in all pairs of interest. On any change it will send a dummy 'tick' to the main EA. So only one extra expert needed besides the main EA.

Edit: or just place this infinite loop in a script...
 

> answer is have eg, EA attached to 10 charts,... but that is not what I'm after
I dont see that as an easy answer but it is the one that allows improved scaling out, i.e. n instances of MT4 (which gives you additional order channel bandwidth)
or n VPS, etc, etc
You could after all have a 'Manager' EA on the most active pair that 'supervises' the others for TP, SL or TS?
So... is there really a compelling reason for <the one> EA?
If so, lets hear it!
-BB-

 
fbj:
1. If a many symbol EA is run [on any chart] and begins applying system on many symbols - we get many orders... duh (bear with me :-)
Since the EA works 24/7 their will be periods when the 'attached to chart' does not get good frequency of data tics. IE, the EA will be unable to effectively manage [in a timely manner] the open trades for other symbols.
So that is my design question - many trades could be open on many symbols and the attached chart which EA runs on may not receive data ticks regularly, meaning the other symbols with trades cannot be managed effectively (eg: trailing S/L either shadow or via ordermodify)
What do you suggest or do about it?

I'll stay out of the broader design question, and assume that you'd really rather stick with the model of a single EA running on a single chart, and that the concern is therefore about ticks drying up.

One possible route which avoids the problems of an infinite loop in start() is to use the technique for triggering calls to start(), as described in topics such as https://www.mql5.com/en/forum/119781. Broadly, you'd do the following:

int start()
{
   ... Normal processing

   // Trigger another call to start()
   int MessageNumber = RegisterWindowMessageA("MetaTrader4_Internal_Message");
   PostMessageA(WindowHandle(Symbol(), Period()), MessageNumber, 2, 0);
   return (0);
}  


There are two problems with this route, however. Firstly, it results in major consumption of processor time. Secondly, it doesn't actually work that reliably. Sooner or later the PostMessage() fails to trigger the start() function, and the EA goes dark until the next real tick - presumably because the message is processed internally by MT4 before the current call to start() has finished, and the message is therefore ignored.

However, a trivially simple DLL might well fix this: the EA calls a DLL function which then sends message to the chart window every second, or every 5 seconds, or whatever, on a timed basis. The EA therefore gets a guarantee that its start() function will be called at least every 1/5/whatever seconds.

 
jjc:
Firstly, it results in major consumption of processor time.

Does it? How bad of a consumption would it be?


Secondly, it doesn't actually work that reliably. Sooner or later the PostMessage() fails to trigger the start() function, and the EA goes dark until the next real tick - presumably because the message is processed internally by MT4 before the current call to start() has finished, and the message is therefore ignored.

But that would also happen if a tick is incoming to the main expert's chart while it's running... So missing ticks that come while start() is running is 'normal' (and is mostly unavoidable). IMHO, I wouldn't say this makes the method 'unreliable'.

 
gordon:

Does it? How bad of a consumption would it be?
[...] IMHO, I wouldn't say this makes the method 'unreliable'.

I've tried it, and it's (a) capable of maxing out a processor core, and (b) simply is unreliable. I can't get it to avoid entering periods where nothing happens until the next real market tick. The point is that, if it sends a simulated tick which gets dropped because it's still in the call to start(), nothing then happens until the next real tick.

The DLL route, however, seems to work perfectly...

 
jjc:

...maxing out a processor core...

Wow. I wouldn't have imagined it's that bad. That's good info. Thanks.


p.s. I just recalled reading about a solution to a similar problem that exists in MT5, see here -> http://mqlmagazine.com/mql-programming/tick-events-in-mql5-complicated-for-now-but-functional/. Just for FYI...

 
gordon:

Just for FYI...

Attached zip file contains the DLL, the source code for the DLL, and an example EA.

Seems to work in terms of doing things like generating simulated ticks every 2 seconds on USDSGD, which usually has much less frequent ticks. Doesn't seem to mind being run on multiple charts at once (and there's no reason why it should). But obviously approach with caution - it's had about 3 minutes testing.

Files:
 
jjc:

Attached zip file contains the DLL, the source code for the DLL, and an example EA.

Cheers!

 
wow... as usual you people certainly wade into the nitty gritty. So much to think about.
jjc - you and your Win code! it just keeps appearing with magical regularity... thanx. like it lots but also tempered by bb's reply.
gordon - thanks for the links, this site has so much info and the 1+2 idea will be looked at - obviously, I do not spend nuff time reading around - maybe the fire is burning out.. oh merde!
bb - my 'issue' is one of h/w resources but yes, how can I actually justify it? Now that I've 'come a cropper' over this, the sick feelings of why did I not consider this at the drawing board stage are manifesting.
.
ok then, think I should setup a full scale test of the 14 pairs interested in with EA on each chart - simple nuff to disable all but one pair per instance via externs.
damn, have really worked hard to make multisym but reading all of above seems to highlight how I've blind-sided self due to the hahaaa coolness of just one EA. The complexity of robust multisym is not imho trival and that in itself should have warned me off such a path. Each day I see on the wall behind the crt the K.I.S.S. logo written on an old ibm 80col card, yet somehow allowed self to stray into complexity.
.
ya, must go away and ponder all the great replies.
.
you dudes are true heroes and your inputs greatly appreciated.
bfn
Reason: