Grouped trades.

 

Hi guys,

I'm trying to implement the following in one of my EA. The concept is having grouped trades. For example:

- I open trade A1 A2 and A3.

- After 1 hour, I open trade B1 B2 and B3

- After 2 hours, some of my trades hit Take profit. For example A1, B2 and B3.

- What I want is to know when all the trades of a group are closed, because maybe I want to open C1, C2 and C3 only when A1,A2 and A3 are closed.

I think what I need is that all the trades in a group have the same magic number. For example all trades of group A will have magic number 1001, and trades of group B will have magic 1002. Then I could check OpenOrders and History Orders to check if all the trades of a group are closed. My main concerns are,

- How do I generate my magic numbers and keep the order even if the EA deinits because of a server failure or something? I could use time based magic number, but what if an order fails? I need to refill the order and then time will be different.

- When I have too many open orders (more than 200), the process can be slow having to check all trades looking for my magic number.

- I've considered the possibility of using a MySQL database to make things easier, but seems like MT4 experienced programmers don't like much this idea. Guess querying a mySQL database on every tick can be problematic.

- I don't like to depend on variables and arrays, because if the EA de-inits, all the information is lost.

- Another option is to use binary or csv files. I would like to use binary, but when testing, it seems I can't overwrite a position in the middle of the file. For example I store doubles in a binary: 300.5|1|300.6|2 and then I want to overwrite the third position and store 300.5|1|300.7|2. I can't get this done, looks like the file just gets messed up.

- I know there are a lot of things in this post, but my main question is, How do you guys deal with an EA that needs to have a "memory" and can continue even after server failure?

 
averied:


- I know there are a lot of things in this post, but my main question is, How do you guys deal with an EA that needs to have a "memory" and can continue even after server failure?

I think you have outlined the main options, determine which is best for you, implement it, test it, fix it, use it . . .
 
averied: How do you guys deal with an EA that needs to have a "memory" and can continue even after server failure?

It depends on how many, what kind of data needs to be handled and in what way.

I think for storing a magic number a good way is using MT4 global variables. Considering server failure (with failing global variables) a good solution may be checking the opened orders and/or history, at least once after EA startup to get the max magicnumber from them and then use GVs (or simple variables in memory) when running the EA.

 

Global variables (not to be confused with globally defined/common variables) will NOT be written to disk in the case of a terminal/OS failure. Only writing a disk file is persistent storage (file, mySQL, etc.)

OP must either use persistent storage or be able to recreate what is needed.

 

Cool! Thanks for the answers.

I would love to have a Database combined with MetaTrader. The thing is I've read that the MySQL wrappers have some problems, as metatrader can't handle well the structures.

Anyway I will give MySQL a try and see if it fits my needs.

However, i would like to ask if there is any known database engine 100% compatible with MetaTrader/mql4. I wouldn't mind any SQL engine, Firebird, MS SQL Server, Access, etc. What do you guys say?

 
WHRoeder:
Global values will NOT be written to disk in the case of a terminal/OS failure. Only writing a disk file is persistent storage.

MT4 Global Variables are also stored in a file: MT4/profiles/gvariables.dat the value is written in the file.

(I checked it just now: I killed the terminal.exe process instantly after creating a GV, after restart it was there. But I would not rely only on this to make a safe solution.)

Data written to a disk file may also be destroyed when OS failure happens before the data is flushed to the disk.

 
averied:

However, i would like to ask if there is any known database engine 100% compatible with MetaTrader/mql4. I wouldn't mind any SQL engine, Firebird, MS SQL Server, Access, etc. What do you guys say?


I have seen sometimes sqlite with MT4. I don't know too much about this: http://www.shmuma.ru/wiki/index.php/Mt4-sqlite

A MySQL wrapper is also available: https://www.mql5.com/en/code/8623

I don't know other MT4 compatible sql engines, I don't know exactly how much compatible are these.

 
averied:
....

- When I have too many open orders (more than 200), the process can be slow having to check all trades looking for my magic number.

- I've considered the possibility of using a MySQL database to make things easier, but seems like MT4 experienced programmers don't like much this idea. Guess querying a mySQL database on every tick can be problematic.

..

- Another option is to use binary or csv files. I would like to use binary, but when testing, it seems I can't overwrite a position in the middle of the file. For example I store doubles in a binary: 300.5|1|300.6|2 and then I want to overwrite the third position and store 300.5|1|300.7|2. I can't get this done, looks like the file just gets messed up.



Why will looping through 200 orders (which although a big number to us, is a small number to computers - if that makes sense) be slower than IO overhead of file access? Is this a known issue with MT4, or just an assumption?


You only need to check you orders if :

1. Bid has hit lowest TP, highest SL for BUYS, or Ask has hit highest TP, lowest SL for SELLS

at which point you:

- loop through the orders etc filtering on magic etc...

- re-compute your next Bid/Ask trigger values.


Also update trigger values when you send a new order...

You may also want to check if Order closed manually but that needs some more logic..

 

Those are nice ideas ydrol, Altough one way or another, getting things done is not a big problem. I'm putting 200 as an example, but the system I'm developing right now is a bit more complicated. It does OK for real time, but for backtesting is very slow.

It's actually complicated because I have nested trades. This means maybe I:

- open a trade, lets call it A1 at 1,2030, I also open another trade B1 at 1,2032

- I open some "cover trades" for A1. So I need to open a trade for example at 1,2025 (A2) 1,2035 (A3) 1,2060 (A4) etc. And I need to do this also for B1 and for other 50trades. So using just standard mql4 this is slow and complicated for backtest, because I have to check how many cover trades I have done for one trade, so I know what is the next cover price, and if we reach this price, then I open the next cover trade.

So this is selectorder for one loop, then do another loop inside the first one and selectorder for cover trades, check price, store the original ticket, etc.. quit slow when I have to cover a big amount of trades, and check the cover trades for each.

So in this case having a database would be very helpful, and I think also much faster for backtesting.

Thanks anyway for the comments. Just wanted to have some extra opinions, but the options seem to be quite clear.

 

A database may help with clearer logic, maybe even with performance (in terms of changing the O(n*n) nested loop lookups in MQ4, to indexed searches, but I'm sure it can all be implemented in MQ4 without "too much" pain...

For each tick you probably have a change in price, but you are also concerned if some orders have closed in the background? And that is why you are re-parsing the orders at each tick?

If no orders have closed then you can re-use values you computed at the last tick?

I'm sure the effort in optimising when to re-parse orders in mq4 will be less than the effort of migrating to a database, and probably provide even better performance than using a database?


As an aside there may also be some value in encoding extra 'metadata' within the magic number. It is 32 bits so give them all a workout!
Reason: