Market Simulation: Position View (V)

Introduction

Hello everyone, and welcome to another article in the series on building a market replay and simulation system.

In the previous article, Market Simulation: Position View (IV), we began connecting the position indicator to three other core components already present in the replay/simulation service. At this initial stage, we will focus on using these four applications on either a demo account or a real account. Even so, I do not recommend using them on a real account yet, because further adjustments are still required. The main reason I do not recommend using these applications on a real account is precisely that the Expert Advisor and the position indicator are not yet stable enough for reliable use.

As for Chart Trade and the Mouse Study indicator, their use on a real account presents no problem. However, Chart Trade by itself, without support from the Expert Advisor, is completely useless.

So, in this article we will discuss a subject that, in most cases, is what a programmer actually does. This differs greatly from what many people imagine a programmer's work to be. When we talk about programming, we spend a great deal of time studying and analyzing data. Only a small part of that time is spent writing code.

Why am I saying this? Because many people imagine that programming consists of continuously writing large amounts of code. In reality, this is done only after analyzing various data and thinking through possible solutions. It is not unusual for a programmer to spend quite a lot of time analyzing files that they may have created themselves in order to understand how everything works. When possible, a debugging session may be used instead of such log files.

However, when the volume of data is large and the analysis takes a long time, it is often preferable to rely on log files. In this article, I will explain how these files can be used to find a possible solution to a problem. To begin, let us examine the existing problems.

The first problems

Despite what was shown in the previous article, the task may appear simple. In reality, several problems remain, along with many tasks that still need to be completed. You, dear reader, may imagine that everything is easy and straightforward. Out of inexperience, you may simply accept whatever is presented to you. And that is a mistake you should try to avoid. Even worse is trying to use something without truly understanding what exactly you are using.

Beginners often pass through a copy-and-paste stage. If you do not want to remain stuck at that stage forever, you should learn how to use certain tools. One of the tools most often used by programmers is documentation. The second is testing, supported by log files.

So, let us examine the next problem we encountered. As shown in the previous article, the Expert Advisor can place the position indicator on the chart. This indicator retrieves position data in order to display the lines corresponding to the position and its price levels on the chart. At first glance, there appears to be no problem, because everything works as expected. However, there are flaws in the code—or, more precisely, in the way the system has been implemented.

Let us assume—as good programmers often do—that the trader removes the Expert Advisor from the chart. Or that the trader tells the Expert Advisor, which was monitoring the full contract, to switch the tracked contract from the full contract to the mini contract. What will happen on the chart in these two cases? Remember that we are only beginning to deal with all of this. In both cases, we will run into problems with the position indicator. In the first case, when the Expert Advisor is removed from the chart, all existing position indicators will remain on the chart. This is incorrect, because we no longer have an Expert Advisor to support the system if the trader wants to close a position. The mechanisms required to close a position through direct interaction with the position indicator have not yet been implemented, although they will be added soon. We need to start thinking about this possibility now. This is because, when this functionality is actually implemented, the system will already be tested for other equally important aspects, such as the case where the Expert Advisor is removed from the chart.

In the second case, when the trader requests a change in the contract type, the failure may be even more serious, because it misleads the trader about what is actually happening in the trading account. The solution in both cases is essentially to remove all position indicators as soon as the Expert Advisor is subjected to any intervention by the trader. Such an intervention causes MetaTrader 5 to generate a DeInit event, which the Expert Advisor handles in OnDeinit. The solution is therefore essentially to replace the old Expert Advisor code with a new version. However, since much of the code needs to be changed, replacing the fragment shown below in the Expert Advisor code will be enough to solve both problems:

54. //+------------------------------------------------------------------+
55. void OnDeinit(const int reason)
56. {
57.     ulong ul;
58.     
59.     switch (reason)
60.     {
61.         case REASON_REMOVE:
62.         case REASON_INITFAILED:
63.             EventChartCustom(0, evEA_At_ChartTrade, -1, 0, "");
64.             break;
65.     }
66.     if (Terminal != NULL) for (int count = PositionsTotal() - 1; count >= 0; count--)
67.     {
68.         ul = PositionGetTicket(count);
69.         if (PositionGetString(POSITION_SYMBOL) != (*Terminal).GetInfoTerminal().szSymbol) continue;
70.         ChartIndicatorDelete(0, 0, IntegerToString(ul));
71.     }
72.     delete Orders;
73.     delete Terminal;
74. }
75. //+------------------------------------------------------------------+

Fragment from the Expert Advisor

Note that we made some changes to this code, as shown on line 63. Originally, an explicit chart ID was supplied when sending custom events. While studying the documentation to reduce the number of checks in the code, I found that when zero is passed as the chart ID, custom events are sent to the current chart from which the call was made. Because of this, we no longer need to check whether the pointer has been initialized. The situation is different on line 66. At this stage, we need to make sure that the pointer has been initialized. If it has not, we ignore the code between lines 66 and 71.

What does this code do? It solves the two problems described above. Note that when there is an open position, the position indicator is displayed on the chart. When the Expert Advisor receives a DeInit event from MetaTrader 5, the function shown in the fragment above is called. On line 68, we retrieve the position ticket by index. Then, on line 69, we check whether the position symbol matches the symbol expected by the Expert Advisor. The Terminal pointer must therefore be initialized. Otherwise, we would not be able to get the correct symbol name.

If the symbol matches the expected one, on line 70 we tell MetaTrader 5 to delete from the chart the indicator whose short name matches the position ticket. "But wait a second. When I open the list of indicators, I do not see any indicator with that name. In other words, there is no indicator whose name matches the position ticket value." You are right, my dear reader. The reason is that the position indicator explicitly defines the short name that MetaTrader 5 uses to identify it. That is exactly the name we must specify here, on line 70.

Excellent, the problem is solved. However, even though this problem has been solved, one overlooked detail can still cause a failure if the trader changes the timeframe while the Expert Advisor is running on the chart. This failure will cause a runtime error, unload the Expert Advisor from the chart, and write the error to the MetaTrader 5 terminal log.

A similar runtime error commonly occurs in C/C++ programs when working with pointers if we try to use a pointer that has not been initialized. "But wait. In the previous article, it was mentioned that pointers are initialized with NULL." Yes, they are initialized with this value. However, when the timeframe is changed, the application is not restarted from scratch; instead, reinitialization begins with OnInit. At that point, a pointer that was initialized only in the global context will no longer be initialized properly.

Thus, the stability of the entire application is put at risk, because we are dealing with a pointer that refers to an invalid or unknown memory area. This is precisely why the use of pointers is restricted in MQL5, to avoid errors like this. But since we really want and need to use pointers, we also need to correct this flaw. However, it is very easy to fix, as shown in the next code fragment.

13. //+------------------------------------------------------------------+
14. input eTypeContract user00 = MINI;         //Cross order in contract
15. //+------------------------------------------------------------------+
16. C_Orders       *Orders;
17. C_Terminal     *Terminal;
18. //+------------------------------------------------------------------+
19. int OnInit()
20. {
21.     Terminal = NULL;
22.     Orders = new C_Orders(0xC0DEDAFE78514269);
23.     
24.     return INIT_SUCCEEDED;
25. }
26. //+------------------------------------------------------------------+

Fragment from the Expert Advisor

Line 21 is what fixes the problem. Compare this fragment with the original code from the last article, and you will understand what changed here. Note that the line numbers remain the same as in the original code. So, three potential errors have been fixed. But we have still not reached the truly difficult problems. To separate the topics a little, let us examine the first of these problems in a new section.

A problem with many solutions

So, everything discussed so far has been aimed at eliminating simple problems that are nevertheless unpleasant because of the failures they can cause. Now we will begin to look at truly difficult problems that need to be solved. The first concerns changes in the average position price. If you are not familiar with this term, I will explain its meaning so that you understand the real problem we are facing.

On HEDGING accounts, the concept of an average price does not exist on the trading server. It applies only from the trader's point of view, because you may have several different positions open, some of them long (Buy), others short (Sell). Each position may have a completely different volume as well as a different opening price. By performing a calculation based on the price and volume of each position, you obtain a certain value. This gives you the average price and volume values. The trader must calculate these values manually if the account is of the HEDGING type.

By contrast, on NETTING accounts the situation is somewhat different. In this case, the average price, as well as the total volume, are calculated and maintained by the trading server. Note the scale of the problem we face when trying to create a replay/simulation service. The project would be much easier if all operations were limited to NETTING accounts. The need to cover both account types creates a series of complications that force us to postpone completion of the project.

What is the actual problem? Unlike opening a new position on a HEDGING account, opening or increasing a position on a NETTING account changes both the average price and the total volume. Handling volume is relatively straightforward, although we have not discussed it yet. Price, however, is somewhat more complex. This is because it is difficult to keep the position indicator tied to a single price. That may sound strange, but consider the following: when the average position price changes on the server, the Expert Advisor receives the update, but the position indicator does not. This is our first problem.

There are other problems, but we will start with this one. You may think: "Fine, all we need to do is make the position indicator follow what is happening with the position. Simple enough." However, dear reader, in practice things are quite different. If you configure the position indicator to track changes in the position, you will have to do it through the OnTimer event, which is strongly discouraged, or through the OnCalculate event, which is also undesirable. I am not saying that you cannot do this. I am simply pointing out that it is undesirable, because it may cause the indicator to freeze. It may also block every other indicator attached to the same symbol chart. In short, this is a very poor solution.

One might then suggest: "We can access the HLINE object and change its position in the Expert Advisor code." This would be a good solution, but it has another problem. The HLINE object is only one of the objects that we will need and use inside the position indicator. Therefore, this solution should be abandoned. The position indicator itself must remain solely responsible for managing its objects.

At this point, you are probably thinking: "You are being stubborn. Why make everything so complicated? Keep it simple." Sometimes I think the same thing, but I enjoy difficult problems. My proposal may give you a headache, because now we are going to dive into a world that will be completely unknown to many beginners. However, for those who earn their living from this, it is a very familiar world. Let us examine how to create and interpret log files. The simplest solution is to use the OnTradeTransaction function to solve our problem, because it is called only when something happens. The server reports exactly what happened, so there is no need for us to investigate it.

But since the OnTradeTransaction function is quite complex, we need a log file to understand what happens when the volume of a position is increased. Pay attention to this. I will show what happens when a position is opened and, shortly after that, its volume is increased. In the end, we close the position completely. All of this is done using the Expert Advisor, the Mouse Study indicator, and the Chart Trade function that we have already developed. For now, we will leave the position indicator aside.

However, the Expert Advisor code we presented is not suitable for our task. It must therefore be replaced with the following version:

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. #property icon "/Images/Market Replay/Icons/Replay - EA.ico"
04. #property description "Demo version between interaction"
05. #property description "of Chart Trade and Expert Advisor"
06. #property version   "1.117"
07. #property link "https://www.mql5.com/pt/articles/13176"
08. //+------------------------------------------------------------------+
09. #include <Market Replay\Order System\C_Orders.mqh>
10. #include <Market Replay\Auxiliar\C_Terminal.mqh>
11. //+------------------------------------------------------------------+
12. enum eTypeContract {MINI, FULL};
13. //+------------------------------------------------------------------+
14. input eTypeContract user00 = MINI;         //Cross order in contract
15. //+------------------------------------------------------------------+
16. C_Orders       *Orders;
17. C_Terminal     *Terminal;
18. //+------------------------------------------------------------------+
19. void AddIndicatorPosition(ulong arg)
20. {
21.     int handle = iCustom(_Symbol, PERIOD_CURRENT, "\\Indicators\\Position View.ex5", arg);
22.     ChartIndicatorAdd(0, 0, handle);
23.     IndicatorRelease(handle);
24. }
25. //+------------------------------------------------------------------+
26. int OnInit()
27. {
28.     Print("Initializing the Expert Advisor...");
29.     Terminal = NULL;
30.     Orders = new C_Orders(0);
31.     
32.     return INIT_SUCCEEDED;
33. }
34. //+------------------------------------------------------------------+
35. void OnTick() {}
36. //+------------------------------------------------------------------+
37. void OnChartEvent(const int id, const long &lparam, const double &dparam, const string &sparam)
38. {
39.     (*Orders).DispatchMessage(id, lparam, dparam, sparam);
40.     switch (id)
41.     {
42.         case CHARTEVENT_CHART_CHANGE:
43.             if (Terminal != NULL) break;
44.             else
45.             {
46.                 ulong ul;
47.                 Terminal = new C_Terminal(0, 0, user00);
48.                 for (int count = PositionsTotal() - 1; count >= 0; count--)
49.                 {
50.                     ul = PositionGetTicket(count);
51.                     if (PositionGetString(POSITION_SYMBOL) != (*Terminal).GetInfoTerminal().szSymbol) continue;
52.                     AddIndicatorPosition(ul);
53.                 }
54.             }
55.         case CHARTEVENT_CUSTOM + evChartTrade_At_EA:
56.             EventChartCustom(0, evEA_At_ChartTrade, user00, 0, "");
57.             break;
58.     }
59. }
60. //+------------------------------------------------------------------+
61. void OnTradeTransaction(const MqlTradeTransaction &trans, const MqlTradeRequest &request, const MqlTradeResult &result)
62. {
63.     Print(__FUNCTION__);
64. 
65.     MqlTradeTransaction ts[1];
66.     ts[0] = trans;
67.     ArrayPrint(ts);
68. 
69.     MqlTradeRequest tr[1];
70.     tr[0] = request;
71.     ArrayPrint(tr);
72. 
73.     MqlTradeResult rs[1];
74.     rs[0] = result;
75.     ArrayPrint(rs);
76. }
77. //+------------------------------------------------------------------+
78. void OnDeinit(const int reason)
79. {
80.     ulong ul;
81.     
82.     switch (reason)
83.     {
84.         case REASON_REMOVE:
85.         case REASON_INITFAILED:
86.             EventChartCustom(0, evEA_At_ChartTrade, -1, 0, "");
87.             break;
88.     }
89.     if (Terminal != NULL) for (int count = PositionsTotal() - 1; count >= 0; count--)
90.     {
91.         ul = PositionGetTicket(count);
92.         if (PositionGetString(POSITION_SYMBOL) != (*Terminal).GetInfoTerminal().szSymbol) continue;
93.         ChartIndicatorDelete(0, 0, IntegerToString(ul));
94.     }
95.     delete Orders;
96.     delete Terminal;
97. }
98. //+------------------------------------------------------------------+

Expert Advisor for creating logs

After looking at this code, dear reader, you may think: "Come on, you have gone crazy. This code does not seem to generate any diagnostic data explaining what happens in the OnTradeTransaction function, does it?" Well, after this code was executed in MetaTrader 5, the following log was produced:

OK      0       12:05:36.509    Expert Advisor (WDO$,M30)       Initializing the Expert Advisor...
HO      0       12:05:48.773    Chart Trade (WDO$,M30)  Send evChartTradeBuy - Args ( 10?WDO$?WDOU23?D?1?0.00?0.00 )
QH      0       12:05:48.773    Expert Advisor (WDO$,M30)           [action] [magic] [order] [symbol] [volume]  [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration]                             [comment] [position] [position_by] [reserved]
FK      0       12:05:48.774    Expert Advisor (WDO$,M30)       [0]        1       0       0 "WDOU23"    1.000 4975.000       0.000 0.00 0.00        1000      0              2           1 1970.01.01 00:00:00 "Order Generated by Experts Advisor."          0             0        ...
QM      0       12:05:48.797    Expert Advisor (WDO$,M30)       OnTradeTransaction
GF      0       12:05:48.797    Expert Advisor (WDO$,M30)           [deal]    [order] [symbol] [type] [order_type] [order_state] [deal_type] [time_type]   [time_expiration] [price] [price_trigger] [price_sl] [price_tp] [volume] [position] [position_by] [reserved]
DS      0       12:05:48.797    Expert Advisor (WDO$,M30)       [0]      0 1612507881 "WDOU23"      0            0             0           0           1 1970.01.01 00:00:00   0.000           0.000      0.000      0.000    1.000          0             0        ...
GQ      0       12:05:48.797    Expert Advisor (WDO$,M30)           [action] [magic] [order] [symbol] [volume] [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration] [comment] [position] [position_by] [reserved]
OL      0       12:05:48.797    Expert Advisor (WDO$,M30)       [0]        0       0       0 ""          0.000   0.000       0.000 0.00 0.00           0      0              0           0 1970.01.01 00:00:00 ""                 0             0        ...
CO      0       12:05:48.797    Expert Advisor (WDO$,M30)           [retcode] [deal] [order] [volume] [price] [bid] [ask] [comment] [request_id] [retcode_external] [reserved]
DL      0       12:05:48.797    Expert Advisor (WDO$,M30)       [0]         0      0       0    0.000   0.000 0.000 0.000 ""                   0                  0        ...
ER      0       12:05:48.797    Expert Advisor (WDO$,M30)       OnTradeTransaction
KK      0       12:05:48.797    Expert Advisor (WDO$,M30)               [deal]    [order] [symbol] [type] [order_type] [order_state] [deal_type] [time_type]   [time_expiration]  [price] [price_trigger] [price_sl] [price_tp] [volume] [position] [position_by] [reserved]
JI      0       12:05:48.797    Expert Advisor (WDO$,M30)       [0] 1191195305 1612507881 "WDOU23"      6            0             0           0           0 1970.01.01 00:00:00 4975.000           0.000      0.000      0.000    1.000 1612507881             0        ...
GI      0       12:05:48.797    Expert Advisor (WDO$,M30)           [action] [magic] [order] [symbol] [volume] [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration] [comment] [position] [position_by] [reserved]
OD      0       12:05:48.797    Expert Advisor (WDO$,M30)       [0]        0       0       0 ""          0.000   0.000       0.000 0.00 0.00           0      0              0           0 1970.01.01 00:00:00 ""                 0             0        ...
CG      0       12:05:48.797    Expert Advisor (WDO$,M30)           [retcode] [deal] [order] [volume] [price] [bid] [ask] [comment] [request_id] [retcode_external] [reserved]
DD      0       12:05:48.797    Expert Advisor (WDO$,M30)       [0]         0      0       0    0.000   0.000 0.000 0.000 ""                   0                  0        ...
EJ      0       12:05:48.797    Expert Advisor (WDO$,M30)       OnTradeTransaction
KQ      0       12:05:48.797    Expert Advisor (WDO$,M30)           [deal]    [order] [symbol] [type] [order_type] [order_state] [deal_type] [time_type]   [time_expiration] [price] [price_trigger] [price_sl] [price_tp] [volume] [position] [position_by] [reserved]
PP      0       12:05:48.797    Expert Advisor (WDO$,M30)       [0]      0 1612507881 "WDOU23"      2            0             4           0           1 1970.01.01 00:00:00   0.000           0.000      0.000      0.000    0.000 1612507881             0        ...
CP      0       12:05:48.797    Expert Advisor (WDO$,M30)           [action] [magic] [order] [symbol] [volume] [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration] [comment] [position] [position_by] [reserved]
CN      0       12:05:48.797    Expert Advisor (WDO$,M30)       [0]        0       0       0 ""          0.000   0.000       0.000 0.00 0.00           0      0              0           0 1970.01.01 00:00:00 ""                 0             0        ...
GH      0       12:05:48.797    Expert Advisor (WDO$,M30)           [retcode] [deal] [order] [volume] [price] [bid] [ask] [comment] [request_id] [retcode_external] [reserved]
HM      0       12:05:48.797    Expert Advisor (WDO$,M30)       [0]         0      0       0    0.000   0.000 0.000 0.000 ""                   0                  0        ...
IS      0       12:05:48.797    Expert Advisor (WDO$,M30)       OnTradeTransaction
OH      0       12:05:48.797    Expert Advisor (WDO$,M30)           [deal]    [order] [symbol] [type] [order_type] [order_state] [deal_type] [time_type]   [time_expiration] [price] [price_trigger] [price_sl] [price_tp] [volume] [position] [position_by] [reserved]
KI      0       12:05:48.797    Expert Advisor (WDO$,M30)       [0]      0 1612507881 "WDOU23"      3            0             4           0           1 1970.01.01 00:00:00   0.000           0.000      0.000      0.000    0.000 1612507881             0        ...
OG      0       12:05:48.797    Expert Advisor (WDO$,M30)           [action] [magic] [order] [symbol] [volume] [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration] [comment] [position] [position_by] [reserved]
GG      0       12:05:48.797    Expert Advisor (WDO$,M30)       [0]        0       0       0 ""          0.000   0.000       0.000 0.00 0.00           0      0              0           0 1970.01.01 00:00:00 ""                 0             0        ...
KR      0       12:05:48.797    Expert Advisor (WDO$,M30)           [retcode] [deal] [order] [volume] [price] [bid] [ask] [comment] [request_id] [retcode_external] [reserved]
LF      0       12:05:48.797    Expert Advisor (WDO$,M30)       [0]         0      0       0    0.000   0.000 0.000 0.000 ""                   0                  0        ...
MI      0       12:05:48.797    Expert Advisor (WDO$,M30)       OnTradeTransaction
CP      0       12:05:48.797    Expert Advisor (WDO$,M30)           [deal] [order] [symbol] [type] [order_type] [order_state] [deal_type] [time_type]   [time_expiration] [price] [price_trigger] [price_sl] [price_tp] [volume] [position] [position_by] [reserved]
PM      0       12:05:48.797    Expert Advisor (WDO$,M30)       [0]      0       0 ""           10            0             0           0           0 1970.01.01 00:00:00   0.000           0.000      0.000      0.000    0.000          0             0        ...
OO      0       12:05:48.797    Expert Advisor (WDO$,M30)           [action] [magic]    [order] [symbol] [volume] [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration]                         [comment] [position] [position_by] [reserved]
EQ      0       12:05:48.797    Expert Advisor (WDO$,M30)       [0]        1       0 1612507881 "WDOU23"    1.000   0.000       0.000 0.00 0.00        1000      0              2           1 1970.01.01 00:00:00 "Order Generated by Experts Advi"          0             0        ...
OM      0       12:05:48.797    Expert Advisor (WDO$,M30)           [retcode]     [deal]    [order] [volume]  [price]    [bid]    [ask] [comment] [request_id] [retcode_external] [reserved]
MP      0       12:05:48.797    Expert Advisor (WDO$,M30)       [0]     10009 1191195305 1612507881    1.000 4975.000 4974.500 4975.000 ""          3531297432                  0        ...
EK      0       12:05:53.264    Chart Trade (WDO$,M30)  Send evChartTradeBuy - Args ( 10?WDO$?WDOU23?D?1?0.00?0.00 )
HG      0       12:05:53.264    Expert Advisor (WDO$,M30)           [action] [magic] [order] [symbol] [volume]  [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration]                             [comment] [position] [position_by] [reserved]
RF      0       12:05:53.264    Expert Advisor (WDO$,M30)       [0]        1       0       0 "WDOU23"    1.000 4974.500       0.000 0.00 0.00        1000      0              2           1 1970.01.01 00:00:00 "Order Generated by Experts Advisor."          0             0        ...
EF      0       12:05:53.287    Expert Advisor (WDO$,M30)       OnTradeTransaction
KM      0       12:05:53.287    Expert Advisor (WDO$,M30)           [deal]    [order] [symbol] [type] [order_type] [order_state] [deal_type] [time_type]   [time_expiration] [price] [price_trigger] [price_sl] [price_tp] [volume] [position] [position_by] [reserved]
EN      0       12:05:53.287    Expert Advisor (WDO$,M30)       [0]      0 1612507912 "WDOU23"      0            0             0           0           1 1970.01.01 00:00:00   0.000           0.000      0.000      0.000    1.000          0             0        ...
KL      0       12:05:53.287    Expert Advisor (WDO$,M30)           [action] [magic] [order] [symbol] [volume] [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration] [comment] [position] [position_by] [reserved]
CI      0       12:05:53.287    Expert Advisor (WDO$,M30)       [0]        0       0       0 ""          0.000   0.000       0.000 0.00 0.00           0      0              0           0 1970.01.01 00:00:00 ""                 0             0        ...
OD      0       12:05:53.287    Expert Advisor (WDO$,M30)           [retcode] [deal] [order] [volume] [price] [bid] [ask] [comment] [request_id] [retcode_external] [reserved]
PI      0       12:05:53.287    Expert Advisor (WDO$,M30)       [0]         0      0       0    0.000   0.000 0.000 0.000 ""                   0                  0        ...
QO      0       12:05:53.287    Expert Advisor (WDO$,M30)       OnTradeTransaction
GF      0       12:05:53.287    Expert Advisor (WDO$,M30)               [deal]    [order] [symbol] [type] [order_type] [order_state] [deal_type] [time_type]   [time_expiration]  [price] [price_trigger] [price_sl] [price_tp] [volume] [position] [position_by] [reserved]
HF      0       12:05:53.287    Expert Advisor (WDO$,M30)       [0] 1191195326 1612507912 "WDOU23"      6            0             0           0           0 1970.01.01 00:00:00 4974.500           0.000      0.000      0.000    1.000 1612507881             0        ...
KD      0       12:05:53.287    Expert Advisor (WDO$,M30)           [action] [magic] [order] [symbol] [volume] [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration] [comment] [position] [position_by] [reserved]
CQ      0       12:05:53.287    Expert Advisor (WDO$,M30)       [0]        0       0       0 ""          0.000   0.000       0.000 0.00 0.00           0      0              0           0 1970.01.01 00:00:00 ""                 0             0        ...
OL      0       12:05:53.287    Expert Advisor (WDO$,M30)           [retcode] [deal] [order] [volume] [price] [bid] [ask] [comment] [request_id] [retcode_external] [reserved]
PQ      0       12:05:53.287    Expert Advisor (WDO$,M30)       [0]         0      0       0    0.000   0.000 0.000 0.000 ""                   0                  0        ...
QG      0       12:05:53.287    Expert Advisor (WDO$,M30)       OnTradeTransaction
GL      0       12:05:53.287    Expert Advisor (WDO$,M30)           [deal]    [order] [symbol] [type] [order_type] [order_state] [deal_type] [time_type]   [time_expiration] [price] [price_trigger] [price_sl] [price_tp] [volume] [position] [position_by] [reserved]
IL      0       12:05:53.287    Expert Advisor (WDO$,M30)       [0]      0 1612507912 "WDOU23"      2            0             4           0           1 1970.01.01 00:00:00   0.000           0.000      0.000      0.000    0.000 1612507881             0        ...
OK      0       12:05:53.287    Expert Advisor (WDO$,M30)           [action] [magic] [order] [symbol] [volume] [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration] [comment] [position] [position_by] [reserved]
OK      0       12:05:53.287    Expert Advisor (WDO$,M30)       [0]        0       0       0 ""          0.000   0.000       0.000 0.00 0.00           0      0              0           0 1970.01.01 00:00:00 ""                 0             0        ...
KE      0       12:05:53.287    Expert Advisor (WDO$,M30)           [retcode] [deal] [order] [volume] [price] [bid] [ask] [comment] [request_id] [retcode_external] [reserved]
LJ      0       12:05:53.287    Expert Advisor (WDO$,M30)       [0]         0      0       0    0.000   0.000 0.000 0.000 ""                   0                  0        ...
ML      0       12:05:53.287    Expert Advisor (WDO$,M30)       OnTradeTransaction
CG      0       12:05:53.287    Expert Advisor (WDO$,M30)           [deal]    [order] [symbol] [type] [order_type] [order_state] [deal_type] [time_type]   [time_expiration] [price] [price_trigger] [price_sl] [price_tp] [volume] [position] [position_by] [reserved]
NR      0       12:05:53.287    Expert Advisor (WDO$,M30)       [0]      0 1612507912 "WDOU23"      3            0             4           0           1 1970.01.01 00:00:00   0.000           0.000      0.000      0.000    0.000 1612507881             0        ...
CR      0       12:05:53.287    Expert Advisor (WDO$,M30)           [action] [magic] [order] [symbol] [volume] [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration] [comment] [position] [position_by] [reserved]
KL      0       12:05:53.287    Expert Advisor (WDO$,M30)       [0]        0       0       0 ""          0.000   0.000       0.000 0.00 0.00           0      0              0           0 1970.01.01 00:00:00 ""                 0             0        ...
GO      0       12:05:53.287    Expert Advisor (WDO$,M30)           [retcode] [deal] [order] [volume] [price] [bid] [ask] [comment] [request_id] [retcode_external] [reserved]
HS      0       12:05:53.287    Expert Advisor (WDO$,M30)       [0]         0      0       0    0.000   0.000 0.000 0.000 ""                   0                  0        ...
IR      0       12:05:53.287    Expert Advisor (WDO$,M30)       OnTradeTransaction
OK      0       12:05:53.287    Expert Advisor (WDO$,M30)           [deal] [order] [symbol] [type] [order_type] [order_state] [deal_type] [time_type]   [time_expiration] [price] [price_trigger] [price_sl] [price_tp] [volume] [position] [position_by] [reserved]
DH      0       12:05:53.287    Expert Advisor (WDO$,M30)       [0]      0       0 ""           10            0             0           0           0 1970.01.01 00:00:00   0.000           0.000      0.000      0.000    0.000          0             0        ...
CJ      0       12:05:53.287    Expert Advisor (WDO$,M30)           [action] [magic]    [order] [symbol] [volume] [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration]                         [comment] [position] [position_by] [reserved]
LE      0       12:05:53.287    Expert Advisor (WDO$,M30)       [0]        1       0 1612507912 "WDOU23"    1.000   0.000       0.000 0.00 0.00        1000      0              2           1 1970.01.01 00:00:00 "Order Generated by Experts Advi"          0             0        ...
CJ      0       12:05:53.287    Expert Advisor (WDO$,M30)           [retcode]     [deal]    [order] [volume]  [price]    [bid]    [ask] [comment] [request_id] [retcode_external] [reserved]
KM      0       12:05:53.287    Expert Advisor (WDO$,M30)       [0]     10009 1191195326 1612507912    1.000 4974.500 4974.000 4974.500 ""          3531297433                  0        ...
JI      0       12:06:04.628    Chart Trade (WDO$,M30)  Send evChartTradeSell - Args ( 11?WDO$?WDOU23?D?2?0.00?0.00 )
GS      0       12:06:04.628    Expert Advisor (WDO$,M30)           [action] [magic] [order] [symbol] [volume]  [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration]                             [comment] [position] [position_by] [reserved]
LR      0       12:06:04.628    Expert Advisor (WDO$,M30)       [0]        1       0       0 "WDOU23"    2.000 4974.000       0.000 0.00 0.00        1000      1              2           1 1970.01.01 00:00:00 "Order Generated by Experts Advisor."          0             0        ...
NS      0       12:06:04.652    Expert Advisor (WDO$,M30)       OnTradeTransaction
DH      0       12:06:04.652    Expert Advisor (WDO$,M30)           [deal]    [order] [symbol] [type] [order_type] [order_state] [deal_type] [time_type]   [time_expiration] [price] [price_trigger] [price_sl] [price_tp] [volume] [position] [position_by] [reserved]
HI      0       12:06:04.652    Expert Advisor (WDO$,M30)       [0]      0 1612508010 "WDOU23"      0            1             0           0           1 1970.01.01 00:00:00   0.000           0.000      0.000      0.000    2.000          0             0        ...
LH      0       12:06:04.652    Expert Advisor (WDO$,M30)           [action] [magic] [order] [symbol] [volume] [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration] [comment] [position] [position_by] [reserved]
LF      0       12:06:04.652    Expert Advisor (WDO$,M30)       [0]        0       0       0 ""          0.000   0.000       0.000 0.00 0.00           0      0              0           0 1970.01.01 00:00:00 ""                 0             0        ...
PQ      0       12:06:04.652    Expert Advisor (WDO$,M30)           [retcode] [deal] [order] [volume] [price] [bid] [ask] [comment] [request_id] [retcode_external] [reserved]
OE      0       12:06:04.652    Expert Advisor (WDO$,M30)       [0]         0      0       0    0.000   0.000 0.000 0.000 ""                   0                  0        ...
RH      0       12:06:04.652    Expert Advisor (WDO$,M30)       OnTradeTransaction
HQ      0       12:06:04.652    Expert Advisor (WDO$,M30)               [deal]    [order] [symbol] [type] [order_type] [order_state] [deal_type] [time_type]   [time_expiration]  [price] [price_trigger] [price_sl] [price_tp] [volume] [position] [position_by] [reserved]
OS      0       12:06:04.652    Expert Advisor (WDO$,M30)       [0] 1191195411 1612508010 "WDOU23"      6            0             0           1           0 1970.01.01 00:00:00 4974.000           0.000      0.000      0.000    2.000 1612507881             0        ...
LP      0       12:06:04.652    Expert Advisor (WDO$,M30)           [action] [magic] [order] [symbol] [volume] [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration] [comment] [position] [position_by] [reserved]
LN      0       12:06:04.652    Expert Advisor (WDO$,M30)       [0]        0       0       0 ""          0.000   0.000       0.000 0.00 0.00           0      0              0           0 1970.01.01 00:00:00 ""                 0             0        ...
PI      0       12:06:04.652    Expert Advisor (WDO$,M30)           [retcode] [deal] [order] [volume] [price] [bid] [ask] [comment] [request_id] [retcode_external] [reserved]
OM      0       12:06:04.652    Expert Advisor (WDO$,M30)       [0]         0      0       0    0.000   0.000 0.000 0.000 ""                   0                  0        ...
RP      0       12:06:04.652    Expert Advisor (WDO$,M30)       OnTradeTransaction
HK      0       12:06:04.652    Expert Advisor (WDO$,M30)           [deal]    [order] [symbol] [type] [order_type] [order_state] [deal_type] [time_type]   [time_expiration] [price] [price_trigger] [price_sl] [price_tp] [volume] [position] [position_by] [reserved]
MI      0       12:06:04.652    Expert Advisor (WDO$,M30)       [0]      0 1612508010 "WDOU23"      2            1             4           0           1 1970.01.01 00:00:00   0.000           0.000      0.000      0.000    0.000 1612507881             0        ...
HF      0       12:06:04.652    Expert Advisor (WDO$,M30)           [action] [magic] [order] [symbol] [volume] [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration] [comment] [position] [position_by] [reserved]
PG      0       12:06:04.652    Expert Advisor (WDO$,M30)       [0]        0       0       0 ""          0.000   0.000       0.000 0.00 0.00           0      0              0           0 1970.01.01 00:00:00 ""                 0             0        ...
DR      0       12:06:04.652    Expert Advisor (WDO$,M30)           [retcode] [deal] [order] [volume] [price] [bid] [ask] [comment] [request_id] [retcode_external] [reserved]
CG      0       12:06:04.652    Expert Advisor (WDO$,M30)       [0]         0      0       0    0.000   0.000 0.000 0.000 ""                   0                  0        ...
FI      0       12:06:04.652    Expert Advisor (WDO$,M30)       OnTradeTransaction
LR      0       12:06:04.652    Expert Advisor (WDO$,M30)           [deal]    [order] [symbol] [type] [order_type] [order_state] [deal_type] [time_type]   [time_expiration] [price] [price_trigger] [price_sl] [price_tp] [volume] [position] [position_by] [reserved]
RN      0       12:06:04.652    Expert Advisor (WDO$,M30)       [0]      0 1612508010 "WDOU23"      3            1             4           0           1 1970.01.01 00:00:00   0.000           0.000      0.000      0.000    0.000 1612507881             0        ...
DM      0       12:06:04.652    Expert Advisor (WDO$,M30)           [action] [magic] [order] [symbol] [volume] [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration] [comment] [position] [position_by] [reserved]
DI      0       12:06:04.652    Expert Advisor (WDO$,M30)       [0]        0       0       0 ""          0.000   0.000       0.000 0.00 0.00           0      0              0           0 1970.01.01 00:00:00 ""                 0             0        ...
HK      0       12:06:04.652    Expert Advisor (WDO$,M30)           [retcode] [deal] [order] [volume] [price] [bid] [ask] [comment] [request_id] [retcode_external] [reserved]
GH      0       12:06:04.652    Expert Advisor (WDO$,M30)       [0]         0      0       0    0.000   0.000 0.000 0.000 ""                   0                  0        ...
JN      0       12:06:04.652    Expert Advisor (WDO$,M30)       OnTradeTransaction
PG      0       12:06:04.652    Expert Advisor (WDO$,M30)           [deal] [order] [symbol] [type] [order_type] [order_state] [deal_type] [time_type]   [time_expiration] [price] [price_trigger] [price_sl] [price_tp] [volume] [position] [position_by] [reserved]
KG      0       12:06:04.652    Expert Advisor (WDO$,M30)       [0]      0       0 ""           10            0             0           0           0 1970.01.01 00:00:00   0.000           0.000      0.000      0.000    0.000          0             0        ...
DQ      0       12:06:04.652    Expert Advisor (WDO$,M30)           [action] [magic]    [order] [symbol] [volume] [price] [stoplimit] [sl] [tp] [deviation] [type] [type_filling] [type_time]        [expiration]                         [comment] [position] [position_by] [reserved]
IH      0       12:06:04.652    Expert Advisor (WDO$,M30)       [0]        1       0 1612508010 "WDOU23"    2.000   0.000       0.000 0.00 0.00        1000      1              2           1 1970.01.01 00:00:00 "Order Generated by Experts Advi"          0             0        ...
LG      0       12:06:04.652    Expert Advisor (WDO$,M30)           [retcode]     [deal]    [order] [volume]  [price]    [bid]    [ask] [comment] [request_id] [retcode_external] [reserved]
RI      0       12:06:04.652    Expert Advisor (WDO$,M30)       [0]     10009 1191195411 1612508010    2.000 4974.000 4974.000 4974.500 ""          3531297434                  0        ...

Operation log file

Does this look inviting? I know this log file looks like a nightmare. It looks like something completely insane and meaningless. But believe me, as I said, we will perform only three operations here. Every value in it has a specific meaning. So, before you give up, let us see exactly how this file was generated. Note that line 01 is exactly line 28 of the Expert Advisor code. In other words, everything printed to the MetaTrader 5 terminal is also written to the log. Despite everything, this is not done by me, but by MetaTrader 5.

Now look at line 02 of the log file. Pay attention to the source of the data: Chart Trade. At that moment, Chart Trade sent a message to the Expert Advisor to open a market position. Please note that this message is repeated two more times. Between these messages, you can see exactly what the Expert Advisor does when working with a real trading server. The displayed data was generated in the OnTradeTransaction function using lines 63, 67, 71, and 75. Let us examine it carefully.

Every time the server sends a response to MetaTrader 5, the name of OnTradeTransaction appears in the log. Notice that there is a certain pattern in the communication. That is, the Expert Advisor sends a request to the server, and the server immediately returns a response. Each response is intercepted by the OnTradeTransaction function, and the structures are printed in the following order: first the data of the MqlTradeTransaction structure, then the data of the MqlTradeRequest structure, and finally the data of the MqlTradeResult structure. This is repeated message after message from the server.

But let us return to the Expert Advisor code to understand how I did this. Note that the structure data is printed using the ArrayPrint function. "How is that possible? Are you really using ArrayPrint to print structure data to the terminal? Come on, you have gone completely crazy. I thought it was necessary to declare all these fields from each structure, but you have really taken the complexity and use of MQL5 to another level. Here I am writing tons of code one line after another, and you show me this."

Well, as I have already said, always choosing the most obvious solution is not very interesting. The beauty lies in pushing the use of the language to a new level. But now the question is whether all these values from the log make any sense to you. If even after this they are still difficult to understand, we can export them to an SQL database and obtain a simpler structure. For this, we would use three different tables inside the database. That, however, is left as an exercise. You did not think I would hand everything to you on a platter, did you?

When you study this data, you will notice that the values of some fields indicate the actions performed by the server. At a certain point, the server generates an order that will be executed at the market price. In this case, the ticket 1612507881 will be created, with volume 1 and operation type zero. In other words, it is a market buy. At this moment, we have no open position. The value in the Position field is zero. This continues until the type field of the MqlTradeTransaction structure takes the value 6. That is, the server sends the TRADE_TRANSACTION_DEAL_ADD enumeration to MetaTrader 5. At that moment, the Position field is no longer zero and takes the same value as the order created on the server.

After that, the server generates event 2, which represents TRADE_TRANSACTION_ORDER_DELETE. At that point, the order created by the server is removed. Then event 3, which represents TRADE_TRANSACTION_HISTORY_ADD, adds the completed operation to the trading history. Finally, event 10 is generated, which represents TRADE_TRANSACTION_REQUEST. This event completes the position opening process. But I want you to pay attention to the value of the created ticket. This is important because, in the next request cycle coming from Chart Trade, this value will appear again. Note that the value of the order created by the server is different. However, the value of this position is preserved.

By the end of the second cycle, we will have a position on the server with a volume of 2 lots. Now we need to close the position. This happens in the third cycle shown in the log. Once again, note that the position ticket value remains unchanged throughout the entire procedure.

An interesting detail appears here: you can see that the server never returned a type 8 value. In this case, that would mean TRADE_TRANSACTION_DEAL_DELETE, which, in this context, would correspond to the second type. This second order type appears quite often precisely because it corresponds to a market order. Thus, we can use this log data to implement the OnTradeTransaction handler, which frees us from constantly scanning the entire list of active orders or open positions in a loop. In the case of positions, this can make no sense at all if we are working on a NETTING account. However, since we want to generalize the management system, the Expert Advisor must be able to correctly handle every possible scenario, although some of them may require a little more work during implementation.

At this point, you may be thinking: "Fine. You have shown all these numbers and ways of obtaining data that we can use to program the Expert Advisor. But I still cannot fully understand how all this will be used in practice to manage and organize communication between the Expert Advisor and the position indicator. In my opinion, what you are doing seems completely meaningless. If I were you, I would do it in a much simpler way." If you think so, I must agree with you. Yes, it would be much easier to do this another way, but I do not want to do the obvious. I want to do it in a completely new, unexplored way that will nevertheless be fully functional in any situation that may arise in the future.

Remember one thing: we are not developing an application that will constantly run in parallel with the trading server. We are developing a system that will simulate the operation of the trading server. This requires a completely different approach to implementing what you are most likely used to seeing in previous implementations. We can no longer use functions such as PositionsTotal to know what is happening. When using the replay/simulation system, we will not be able to rely on the help or support of these functions. We need to choose completely different paths from those that many people consider the most reasonable course of action.

Final thoughts

Some time ago, I wrote an article on how to use the OnTradeTransaction function. Since then, much has changed in my understanding of MetaTrader 5 and MQL5. That is why I can now demonstrate nuances that I had not planned to discuss at the time. This is because I have noticed growing interest in this topic among beginner developers and enthusiasts who are trying to understand why my code looks so unconventional. I sincerely hope that these articles help everyone understand that we can do much more than many people think possible.

Now that the first of the major problems to be solved has been presented, in the next article we will be able to run the system more fully and create Expert Advisor code that will allow the position indicator to work more reliably. Since the planned implementation requires deep and detailed explanations, so that you can clearly understand the mechanics of all the processes, I will not include in this article the modified Expert Advisor code responsible for automatically deleting the position indicator when the position is closed or initializing it on the chart when a trade is opened. Although doing this is not nearly as difficult as it may seem at first glance.

The real challenge is to understand why this code works successfully without using the elements you are used to seeing in ordinary code. But we will talk about that in the next article. For now, you can study what we have covered in this article, because programming is nothing more than understanding a problem and finding its mathematical solution.

File	Description
Experts\Expert Advisor.mq5	Demonstrates interaction between Chart Trade and the Expert Advisor (Mouse Study is required for interaction).
Indicators\Chart Trade.mq5	Creates the window for configuring the order to be sent (Mouse Study is required for interaction).
Indicators\Market Replay.mq5	Creates controls for interacting with the replay/simulation service (Mouse Study is required for interaction).
Indicators\Mouse Study.mq5	Provides interaction between graphical controls and the user (required both for the replay/simulation service and for the real market).
Indicators\Order Indicator.mq5	Responsible for placing market orders, providing interaction with them and control over them.
Indicators\Position View.mq5	Responsible for identifying market positions, providing interaction with them and control over them, providing interaction with them and control over them.
Services\Market Replay.mq5	Creates and maintains the market replay/simulation service (the main file of the entire system).