MQL's OOP notes: Singleton, Command queue, and Template method patterns in simple order manager
Introduction
The errors can be divided into 2 distinct categories: permanent (critical) and temporary (manageable). Example of the 1-st kind could be ERR_NOT_ENOUGH_MONEY and of the 2-nd - ERR_OFF_QUOTES. The difference between them is that temporary errors can vanish themselves after one or more retries and refreshes of quotes, while critical errors can be solved only by deliberate change in trading function parameters or trading environment (for example, trader could deposit more money on his account or change its leverage).
Most of MQL programmers try to handle temporary errors by running a loop with a trading function call and refreshing rates until it succeeds or a predefined number of iterations achieved. This appoach works but has many drawbacks. If you retry the call too often, this may overload you local trade context (which is only one for the terminal) or your server (which can give you another error - too many requests). And if you call Sleep function in between, your expert will miss any other events (such as chart events) or receive them with a lag.
The logic of using commands is asynchronous. We'll create a command, pass it to a manager (which we'll discuss below) for execution, and then allow our code to do some other tasks (if any) or keep idling until the command will be finally completed or failed, and then the manager will notify our code about result.
The command
class OrderCommand { protected: int cmdID; datetime created;
There should be a way to identify every command, this is why we introduce cmdID and created fields.
The ID can be assigned by incrementing a global (hence static) counter in the command construtor (see below).
static int cmdIncrementedID;
The variable cmdIncrementedID must be initialized - here we start ID numbering from 1.
int OrderCommand::cmdIncrementedID = 1;
Every command can have custom deviation and magic number, though we'll provide another common place for these settings later.
string symbol; int cmd; double volume; double price; double stoploss; double takeprofit; string comment; datetime expiration; color arrow_color;
As far as the command is asynchronous let's provide a field for resulting value and public method to access it:
int getResult() const { return result; }
The command should handle errors, specifically "decide" if an error can be fixed by retry or not. For that purpose we add an array with temporary error codes.
static int HandledErrorCodes[];
and the function for checking this array:
int handledError(string opName, int errCode, bool &priceRequote) { int handled = -1; priceRequote = false; for(int i = 0; i < ArraySize(HandledErrorCodes); i++) { if(HandledErrorCodes[i] == errCode) { handled = i; priceRequote = (i >= 7); break; } } Print(opName + " error: ", errCode, " handled:", (handled != -1)); return handled; }
The array should be filled after the class definition:
int OrderCommand::HandledErrorCodes[] = {0, ERR_SERVER_BUSY, ERR_NO_CONNECTION, ERR_TRADE_TIMEOUT, ERR_OFF_QUOTES, ERR_BROKER_BUSY, ERR_TRADE_CONTEXT_BUSY, ERR_INVALID_PRICE, ERR_PRICE_CHANGED, ERR_REQUOTE};
The errors starting from 7-th position (2-nd line) implies that price refreshing may help.
Now we can code the constructor:
public: OrderCommand(string symbol = "", int cmd = -1, double volume = 0, double price = 0, int deviation = 0, double stoploss = 0, double takeprofit = 0, string comment = "", int magicId = 0, datetime expiration = 0, color arrow_color = CLR_NONE) { this.symbol = symbol == "" || symbol == NULL ? _Symbol : symbol; int d = (int)MarketInfo(this.symbol, MODE_DIGITS); this.cmd = cmd; this.volume = volume; this.price = NormalizeDouble(price, d); this.stoploss = NormalizeDouble(stoploss, d); this.takeprofit = NormalizeDouble(takeprofit, d); this.expiration = expiration; this.comment = comment; this.deviation = deviation; this.magicId = magicId; cmdID = cmdIncrementedID++; created = TimeCurrent(); }
All fields are assigned and ID is incremented for every new object. The fields are similar to properties of MT4 orders, so please consult with documentation if necessary.
A method for reading every field value should be implemented, for example:
int getCmdID() const { return cmdID; } static int getNextCmdID() { return cmdIncrementedID; } ... double getVolume() const { return volume; }
Now let us consider error handling functions provided by MT4 and try to make use of them. Most impostant thing is that GetLastError function not only returns last error code but also clears it internally, so that successive calls will return 0. I don't think this is a good design. For example, if you use a 3-d party library which calls GetLastError inside, your code will never detect which error has actually happened. So let us save error code in the object as integer _LastErrorOverride and implement the wrapper function:
int getLastError() { if(_LastErrorOverride != 0) return _LastErrorOverride; int e = GetLastError(); if(e != 0) { _LastErrorOverride = e; } return _LastErrorOverride; }
The function returns last error code from the variable if it's assigned, or requests the code from the built-in GetLastError and then saves result in the variable. Also we need to wrap another standard function:
void resetLastError() { ResetLastError(); _LastErrorOverride = 0; }
After all the preparations we've made by abovementioned code, we are ready to deal with main task of any command - its execution. Our base class should provide a method which establishes some default process in a step-by-step manner. Derived classes will be able to customize every step. This design pattern is called "template method".
For the simplicity we'll use the former approach with a loop - this is harmless as far as we can control it by parameters and even make as trivial as single pass with zero sleeping time. Inside the loop there will be the following steps:
- reset error code;
- execute command specific action;
- analyze error code;
- if error is temporary - retry (go to step 1), otherwise - stop the process with the error.
int process(const int maxRetries, const int timeWaitMsecs) { int i; for(i = 0; i < maxRetries; i++) { resetLastError(); int code = execute(); if(code > 0) return code; // success Print("DATA:", symbol, " ", cmd, " p=", D2S(price), " sl=", D2S(stoploss), " tp=", D2S(takeprofit), " / ask=", D2S(MarketInfo(symbol, MODE_ASK)), " bid=", D2S(MarketInfo(symbol, MODE_BID)), " V=", D2S(volume), " s=", (int)MarketInfo(symbol, MODE_SPREAD), " ", (int)MarketInfo(symbol, MODE_STOPLEVEL)); int err = getLastError(); bool priceRequote = false; if(handledError(classname(), err, priceRequote) == -1) break; Sleep(timeWaitMsecs); // NB: doesn't work in the tester RefreshRates(); if(cmd == OP_BUY) price = MarketInfo(symbol, MODE_ASK); else if(cmd == OP_SELL) price = MarketInfo(symbol, MODE_BID); // we can't deduce updated price for pending orders and order deletion if((cmd > OP_SELL && priceRequote) || cmd < 0) break; Print("New price=", D2S(price)); } if(i == maxRetries) { Print("Max retries count ", maxRetries, " reached, give up"); return 0; // neutral, retry later } return -1; }
The loop runs maxRetries times. On every interation we start from resetting error code and then call yet unknown method execute which returns an integer number. This method should be virtual and overridden in descendants. This is where every specific command will perform its actual task, such as opening or closing an order. The resulting value should be positive on success, negative on error and 0 in a neutral situation. If it's positive we exit the method immediately returning the code (the meaning of the code may depend from specific command). Otherwise we output command details in log and read last error code by means of getLastError. Next we use handledError (described several paragraphs above) to discover if the error can be handled by waiting a bit or not. If not, we break the loop and exit with result -1. If the error seems temporary, we Sleep for timeWaitMsecs period, refresh rates and update command's price if it's for new market order. Then the cycle repeats if attempts count does not exceed maxRetries. If it is, we return 0, signalling outside that the command is not completed yet, but there are chances to make it next time.
In the call of handledError there is another yet unknown virtual function classname. Both virtual functions - execute and classname - can be made pure abstract in the base command class or provide minimal default implementation, for example:
virtual int execute() // = 0; { _LastErrorOverride = ERR_NO_RESULT; return -1; } virtual string classname() const // = 0; { return typename(this); }
Execute does actually nothing and returns error (because we don't want to run the dummy process again and again).
Classname returns a name of current class. You'll see that we will override the function in all descendants with very same implemenation - so in every case typename will return actual name of a derived class.
class OrderCreateCommand: public OrderCommand { public: OrderCreateCommand(int cmd, double volume, string symbol, double price = 0, int deviation = 0, double stoploss = 0, double takeprofit = 0, string comment = "", int magicId = 0, datetime expiration = 0, color arrow_color = CLR_NONE): OrderCommand(symbol, cmd, volume, price, deviation, stoploss, takeprofit, comment, magicId, expiration, arrow_color) { } virtual int execute() { if(price == 0) // use current price for market/instant orders { if(cmd == OP_BUY) price = MarketInfo(symbol, MODE_ASK); else if(cmd == OP_SELL) price = MarketInfo(symbol, MODE_BID); } result = OrderSend(symbol, cmd, volume, price, deviation, stoploss, takeprofit, comment, magicId, expiration, arrow_color); return result; } virtual string classname() const { return typename(this); } };
This is a simplified version of the code. At the bottom you'll find a file with complete code attached.
And here is how a command for order modification can look like.
class OrderModifyCommand: public OrderCommand { private: int ticket; public: OrderModifyCommand(int ticket, double price = 0, double stoploss = 0, double takeprofit = 0, datetime expiration = 0, color arrow_color = CLR_NONE): OrderCommand("", -1, 0, price, 0, stoploss, takeprofit, NULL, 0, expiration, arrow_color) { this.ticket = ticket; if(OrderSelect(ticket, SELECT_BY_TICKET)) { if(this.price == 0) this.price = OrderOpenPrice(); this.symbol = OrderSymbol(); this.cmd = OrderType(); this.volume = OrderLots(); this.comment = OrderComment(); this.magicId = OrderMagicNumber(); } } virtual int execute() { result = OrderModify(ticket, price, stoploss, takeprofit, expiration, arrow_color); return result; } virtual string classname() const { return typename(this); } };
You'll find all the other commands in the attached file as well.
The manager
class OrderManager { private: int slippage; int magic; int maxRetries; int timeWaitMsecs;
The manager should store all pending commands somewhere.
OrderCommand *commands[];
And the manager should be most likely instantiated only once. In fact, there could be cases when you want to have many managers in a single expert adviser, but for the simplicity we decide to use only one instance in this framework. This is why we use the "singleton" design pattern.
static OrderManager *manager; // singleton object instance OrderManager() // constructor is hidden from outer space { maxRetries = 10; timeWaitMsecs = 5000; slippage = 10; manager = GetPointer(this); } public: static OrderManager *getInstance() { if(manager == NULL) manager = new OrderManager(); return manager; }
The manager can only be created by the public factory method getInstance, which either creates a new manager object (if it does not yet exist) or returns already existing one.
As you remember the manager stores deviation and magic number, which are default values for all commands. The manager can "publish" them via getters:
int getDeviation() const { return slippage; } int getMagic() const { return magic; }
As a result, we can use the default values in OrderCommand constructor and thus allow a caller to omit these specific properties in constructor call.
// updated lines in the OrderCommand constructor: // use default values if deviation or magic are not specified this.deviation = deviation > 0 ? deviation : OrderManager::getInstance().getDeviation(); this.magicId = magicId > 0 ? magicId : OrderManager::getInstance().getMagic();
But lets go back to the manager.
To initiate execution of a command we pass its object (pointer) to the manager.
void place(OrderCommand *cmd) { int i, n = ArraySize(commands); for(i = 0; i < n; i++) { if(commands[i] == NULL) break; } if(i == n) { ArrayResize(commands, n + 1); } else { n = i; } commands[n] = cmd; }
The method finds an empty element (if any) in the array of commands or extends the array and reserves new empty slot for the command. The commands stored in the array should be processed in a dedicated method.
void processAll() { int n = ArraySize(commands); for(int i = 0; i < n; i++) { if(CheckPointer(commands[i]) != POINTER_INVALID) { int r = commands[i].process(maxRetries, timeWaitMsecs); if(r != 0) // succeeded or failed, otherwise 0 - still pending { commands[i] = NULL; } } } }
For every not yet completed command it calls its method process, and if it fails unrecoverably or succeeds the command is removed from the queue.
The method processAll should be called periodically, for example from a timer event. For that purpose we'll define:
void onTimer()
{
processAll();
}
which should be invoked from standard OnTimer function of EA. Other important events should be also utilized.
void onInit() { // TODO: restore command queue from Global Variables } void onDeinit() { // TODO: serialize pending commands to Global Variables // TODO: clean up the queue if(CheckPointer(&this) == POINTER_DYNAMIC) delete &this; }
Lets wire up all things together using new order creation as example.
To open a new order we have to create an object of the class OrderCreateCommand and pass it to the manager via its place method. Next time the timer event fires, the manager executes all pending commands including our OrderCreateCommand. This will try to do OrderSend, error analysis, and then either leave the command in the queue for more trials or exclude the command as completed or failed (depending from the result).
Refinement
The first issue is not an issue if we save the pointer to our command in the EA an delete it ourselves. But this is not convenient in many cases. For example, it's much simplier to have an option to write:
manangerObject.place(new OrderCreateCommand(OP_BUY, 0.1, Symbol()));
and "forget" about the object allowing the manager to release the memory.
As far as use-cases may vary, automatic deletion of processed commands should be enabled or disabled by a flag. For this purpose let's add a protected variable selfDestruction to OrderCommand class and corresponding accessor method:
bool isSelfDestructing() { return selfDestruction && (CheckPointer(&this) == POINTER_DYNAMIC); }
Here we added the check if the pointer is dynamic, because only dynamically allocated objects can be deleted explicitly (we can't guarantee that some programmer will not set selfDestruction to true mistakenly for an automatically allocated object).
The variable can be initialized through a constructor parameter (in OrderCommand or in derived classes), so that caller can specify if he/she wants the object to devastate itself. Actually we can provide helper constructors with shorter syntax, for example:
OrderCreateCommand(const int cmd, const double volume): OrderCommand(_Symbol, cmd, volume) { selfDestruction = CheckPointer(&this) == POINTER_DYNAMIC; // convenient but tricky implicit initialization }
This way we have no need to pass additional parameter to the constructor. You may look through the attached code for more possibilities, including the "builder" pattern (which allows you to setup object properties on demand step by step, by chainable simple methods).
Once the variable selfDestruction is assigned, we can use it for object "finalization". In the manager's processAll method change the lines to:
if(r != 0) // succeeded or failed, can be deleted { if(commands[i].isSelfDestructing()) { delete commands[i]; } commands[i] = NULL; }
In the attached file this part is more eleborated, but it's simplified here for readability. In any way, we can now delegate destruction of the command objects to the framework.
The second issue was related to returning result of a command execution back to EA. When it comes to asyncronous commands the best way for passing information is a notification interface.
class OrderCallback { public: virtual void onSuccess(const int result, const OrderCommand *cmd) // = 0; { Print("Completed ", cmd.toString(), ", result: ", result); } virtual void onFailure(const int error, const OrderCommand *cmd) // = 0; { Print("Failed ", cmd.toString(), ", error: ", error); } virtual void onTimeout(const int timeout, const OrderCommand *cmd) // = 0; { Print("Timeout ", cmd.toString(), ", timeout: ", timeout); } };
Again, it could be a pure interface, that is an abstract class with pure virtual methods and no data, but we provide a minimal base implementation. The methods allows us to send 3 notifications: on successfull task completion, on insolvable problem, and on timeout (the usage of the later one is shown in the code only).
The variable of the type OrderCallback should be added to the manager class.
private:
OrderCallback *pCallback;
Then EA can implement an object of this class and assign it via special method in the manager:
void bindCallback(OrderCallback *c)
{
pCallback = c;
}
Now we can elaborate processAll method by adding the notifications:
void processAll() { int n = ArraySize(commands); for(int i = 0; i < n; i++) { if(CheckPointer(commands[i]) != POINTER_INVALID) { int r = commands[i].process(maxRetries, timeWaitMsecs); if(r != 0) // succeeded or failed, otherwise 0 - still pending { if(CheckPointer(pCallback) != POINTER_INVALID) { if(r > 0) { pCallback.onSuccess(r, commands[i]); } else // r < 0 { pCallback.onFailure(commands[i].getLastError(), commands[i]); } } if(commands[i].isSelfDestructing()) { delete commands[i]; } commands[i] = NULL; } } } }
As you see, successfully completed command will send back the result of its execution. For OrderCreateCommand this is a ticket, and for other commands the content will vary. In case of a fatal error its code is sent with the notification.
Finally, for convenience we can add shorthand methods which will mimic standard functions such OrderSend, OrderModify etc. Here is a helper method in the manager:
int Send(string symbol, int cmd, double volume, double price, int deviation, double stoploss, double takeprofit, string comment = "", int magicId = 0, datetime expiration = 0, color arrow_color = CLR_NONE) { OrderCreateCommand create(cmd, volume, symbol, price, deviation, stoploss, takeprofit, comment, magicId, expiration, arrow_color); return create.process(maxRetries, timeWaitMsecs); }
And accompanying define:
#define OrderSend OrderManager::getInstance().Send
More features are actually implemented in the code of the manager, but they are left uncovered in the text due to the lack of space. For example, it's possible to try to execute commands synchronously (as shown above). You may inspect the source code for details.
The command with delays
class OrderCreateCommandWithDelay: public OrderCreateCommand { private: int delay; // seconds public: OrderCreateCommandWithDelay(int delay, int cmd, double volume): OrderCreateCommand(cmd, volume) { this.delay = delay; if(delay > 0) Print("delay till ", (string)(created + delay)); } virtual int execute() { if(created + delay > TimeCurrent()) { Print("delay"); // randomly switch between handled (ERR_BROKER_BUSY) and unhandled (ERR_MARKET_CLOSED) simulated error this._LastErrorOverride = MathRand() > 32767 / 2 ? ERR_MARKET_CLOSED : ERR_BROKER_BUSY; return -1; } // after the specified delay, execute order as usual if(price == 0) { if(cmd == OP_BUY) price = MarketInfo(symbol, MODE_ASK); else if(cmd == OP_SELL) price = MarketInfo(symbol, MODE_BID); } result = OrderSend(symbol, cmd, volume, price, deviation, stoploss, takeprofit, comment, magicId, expiration, arrow_color); return result; } virtual string classname() const { return typename(this); } };
Look how this code works according to testing logs.
1-st line was issued by the command constructor at the moment when EA decided to open a trade. The 2-nd indicates that a simulated delay was applied once. The line 4 shows the code of simulated error - 137 (busy), and marks it as handled, that is the framework will retry this operation automatically. New price is received in the line 5, and second iteration of retries is logged in line 6. Since this is the 2-nd attempt and maxRetries was set to 2 processing is paused at this tick, according to the line 10. On an another tick (line 11), which is not actually the next tick, because commands are executed with the given timer period (1 second in the example setup) we see error 138 (requote), which is generated by the tester because the command object tries to open new order using saved price. This is also a handled error, so the framework refreshes quotes and get new price (line 14), after which the order is successfuly opened (line 15), and the framework sends notification about this in the line 16.
The Expert Adviser
It works. It has been tested on EURUSD H1. You may compare execution without and with delays. You may also try to embed the framework in your existing EA since it supports regular syntax of OrderXXX functions and synchronous execution. Yet its advantage is asynchrony, and to enable it you need only to add ASYNC_ORDERS define before include. In this case, the manager will try to execute commands instantly and return their appropriate results, but if a minor error occurs, the command will be added to the queue for execution in future. If you need to get asynchronous results (such as ticket number), override notification callbacks as appropriate.
This is a draft rather than a real life expert adviser, but it covers many practical aspects making it a good starting point for further development.
