Developing a Replay System (Part 66): Playing the service (VII)

Introduction

In this article, we will begin by doing something a little different. But first, let's briefly recap what was covered in the previous article "Developing a Replay System (Part 65): Playing the service (VI)". There, we addressed the issue related to the percentage indicator provided by the mouse pointer. Previously, during replay or simulation using this indicator, an incorrect value was being displayed. In addition, we implemented a system that enables fast-forwarding, allowing us to jump to a specific point without having to wait several minutes or even hours in some cases. However, before continuing with this article, I would like you to watch the video below. This will help you better understand the points I will explain at the beginning of this article.

Something to watch...

I think the video speaks for itself. Still, I would like to explain a bit about what is happening. This replay/simulation application relies heavily on the use of a customized asset. That much is clear, and I believe everyone agrees on this point. However, there are failures that are not caused by the programming we are doing. These failures have different origins. Some are unusual, and others occur almost randomly. When the cause of the issue can be identified, we are able to address it. However, when the problem occurs randomly, the situation changes, and we must learn to work around it.

At the time of writing this article, the latest version of the MetaTrader 5 platform is exactly the one shown in the video. I am not certain where the problem lies, but it clearly exists, and you have likely already encountered it or will eventually encounter in the future. Until the issue demonstrated in the video is definitively resolved, you will need to adapt. In other words, before loading the replay or simulation application, configure it exactly the way you intend to use it throughout the session. This way, you will avoid the need to change the chart timeframe, which is precisely when MetaTrader 5 strangely loses contact with the ticks or bars in the customized asset. Perhaps by the time you are reading this article, the issue shown in the video has already been fixed. If so, great. If not, follow the advice I just provided to avoid an unpleasant experience when using the replay/simulation application.

With that said, we can now move on to the work ahead in this article. In the previous article, one outstanding issue remained. Since it is more immediate, we will begin with it.

Implementing the Remaining Time Until the Next Bar

One feature that many financial market traders appreciate and frequently monitor is the amount of time remaining until the next bar begins. At first, this might seem trivial or like a waste of time. However, there are operational models where this information is crucial, as traders often place their orders just seconds before the next bar starts.

In certain timeframes or simply through the trader's experience they develop an intuitive sense of how much time remains. However, traders who are just starting out do not yet possess this skill. Therefore, it becomes necessary to provide them with some way to be informed about it. In live markets, this is quite easy to do, since we are always connected to the trading server (on a demo or real account). In addition, the fact that operations are constantly evolving makes it even easier to implement this indication, since time does not stand still. But it is this last aspect that complicates things a little when applying replay/simulation. The fact that the user can pause the application indefinitely or even get to a position where the bars are at the beginning or almost at the end makes things much more complicated. It becomes so challenging that we will need a real juggling act to avoid creating a Frankenstein solution just to display the remaining time of the bar.

Before proceeding, let’s review what we already have. In the previous article, we modified the mouse indicator code so that we now use a different version of the OnCalculate event handler. This change, in fact, proves to be quite useful in this case. The reason for this is that it provides us with an array that will store the time value. With just a small modification, we will get the snippet shown below:

44. //+------------------------------------------------------------------+
45. int OnCalculate(const int rates_total, const int prev_calculated, const datetime& time[], const double& open[],
46.                 const double& high[], const double& low[], const double& close[], const long& tick_volume[], 
47.                 const long& volume[], const int& spread[]) 
48. {
49.    Print(TimeToString(time[rates_total - 1], TIME_DATE | TIME_SECONDS)); // To Testing ...
50.    GL_PriceClose = close[rates_total - 1];
51.    m_posBuff = rates_total;
52.    (*Study).Update(m_Status);   
53.    
54.    return rates_total;
55. }
56. //+------------------------------------------------------------------+

Source code fragment of Mouse Study.mq5

Notice that a new line has been added. Line 49 will allow us to view the latest value found in the time array. Now, pay close attention to an important detail. The value in this array has been adjusted to ensure that the one-minute bar is constructed properly. i.e. within the one-minute window.

If you run the replay/simulation application with this new code present in the mouse indicator, you will see information in the toolbox very similar to what is shown in the animation below:

Very well, you can observe that the mouse indicator is printing information each time the OnCalculate function is called, but the value in terms of seconds is not changing. So, thinking carefully, you might wonder: What if we inserted a value in seconds into this information? Could we then calculate how much time remains until the next bar appears? If you thought along these lines, it means you have understood what exactly we will need to do to create this feature. So, let's test this idea. To do that, we will need to make a small change in the header file C_Replay.mqh. The relevant code snippet can be seen below.

69. //+------------------------------------------------------------------+
70. inline void CreateBarInReplay(bool bViewTick)
71.          {
72.             bool    bNew;
73.             double dSpread;
74.             int    iRand = rand();
75. 
76.             if (BuildBar1Min(m_Infos.CountReplay, m_Infos.Rate[0], bNew))
77.             {
78.                m_Infos.tick[0] = m_MemoryData.Info[m_Infos.CountReplay];
79.                if (m_MemoryData.ModePlot == PRICE_EXCHANGE)
80.                {                  
81.                   dSpread = m_Infos.PointsPerTick + ((iRand > 29080) && (iRand < 32767) ? ((iRand & 1) == 1 ? m_Infos.PointsPerTick : 0 ) : 0 );
82.                   if (m_Infos.tick[0].last > m_Infos.tick[0].ask)
83.                   {
84.                      m_Infos.tick[0].ask = m_Infos.tick[0].last;
85.                      m_Infos.tick[0].bid = m_Infos.tick[0].last - dSpread;
86.                   }else if (m_Infos.tick[0].last < m_Infos.tick[0].bid)
87.                   {
88.                      m_Infos.tick[0].ask = m_Infos.tick[0].last + dSpread;
89.                      m_Infos.tick[0].bid = m_Infos.tick[0].last;
90.                   }
91.                }
92.                if (bViewTick) CustomTicksAdd(def_SymbolReplay, m_Infos.tick);
93.                m_Infos.Rate[0].time = m_MemoryData.Info[m_Infos.CountReplay].time; //< To Testing...
94.                CustomRatesUpdate(def_SymbolReplay, m_Infos.Rate);
95.             }
96.             m_Infos.CountReplay++;
97.          }
98. //+------------------------------------------------------------------+

C_Replay.mqh source code snippet

Look how interesting and simple it is to implement. All we had to do was add line 93 to the code snippet to make it possible to include the value in seconds. It couldn't be simpler than that. But will this actually work? Remember the following fact: the CustomRatesUpdate function documentation states that the time value, for which the rates should be created, must fall within the one-minute window. You might then think, this isn't a problem, since we're not changing the window; we're just adding the seconds value, and most likely, this will be ignored by the function but passed to the mouse indicator so that it can be noticed in the OnCalculate function. In some ways, I agree with your theory, but until we actually test the hypothesis, it remains just a theoretical hypothesis. So, we compiled the code, and the result is what you can see in the video below:

A quick demonstration

But what was that? What on earth happened? To be honest, I didn't expect this to happen. I thought it would work. In fact, the idea isn't entirely wrong: you have probably noticed that the mouse indicator gave us what we wanted. However, the chart content... Well, it's better to find another way to approach this. But you should have noticed that we can still do some interesting things. In fact, I used this same idea to create an indicator in the past. You can see this indicator, at least the open version of it, in the article Developing a trading Expert Advisor from Scratch (Part 13): Time and Trade (II). That version is now obsolete, but it helps to understand what happened here. And before anyone asks: NO, I will not sell, give, lend, or show the current version of that indicator. It is for personal use only.

Alright. I believe you now understand how things work and why something that seems simple actually requires us to perform more intricate manipulation to achieve the desired results.

You might be thinking that we need to come up with a new way of doing things. But the solution already existed in the version that used terminal global variables. However, we will no longer use that same solution. We need to be creative and truly understand how programs pass information between themselves. We're already doing this by using the control indicator to manage the service, and the service reports back to the indicator on our position. The same approach will need to be applied here but with one important detail: we cannot trigger custom events every second - or worse, every tick - to inform the mouse indicator of our current time position. If this is done, we will eventually degrade the application performance, forcing us to further reduce the maximum number of ticks per minute. This is something I want to avoid at all costs. However, we do need some synchronization between the information in the ticks and the information related to the current bar's time. This is the main concern. So, let's undo the changes seen in the code snippets and focus on trying to create a different solution that will actually address the problem. But that will be the topic for the next section.

Thinking of Another Way to Display the Remaining Time

Unlike what happens when we are connected to a live trading server, handling this within the replay/simulator environment is far more complicated, especially considering how I want to approach it. I do not intend to use terminal global variables to transfer information. The fact is that when we are connected to a live server, all clocks need to stay synchronized, at least down to the second. You might not fully understand how significantly this impacts everything, but to perform any checks based on remaining seconds, all you really need is to know how many seconds are left until a given time. When connected to a live server, this is very simple, because we can use the system's internal clock to perform this calculation and get the remaining time.

However, when using the replay/simulation environment, everything becomes much more complex. The reason for this increased complexity lies precisely in the nature of what we're doing: replaying or simulating. But how can simply running a replay or simulation make it so much harder to know the time remaining until the next bar? That doesn't make sense. Indeed, if you look at the issue superficially, it really doesn't make sense that replaying or simulating data makes it so much more difficult to determine how much time is left before a new bar begins. But the entire issue revolves around one key point: being able to read the clock correctly.

Let's consider the following: suppose the first one-minute bar to be plotted on the chart actually takes a full minute to be plotted. Alright. That's the first condition for understanding the issue: the bar is fully drawn over one minute. Now, if you start the application so that plotting begins exactly at second zero, then every system minute will bring a new bar. Perfect, problem solved. This is because synchronization is perfect. So, all we would need to do is check the system clock, and we'd know exactly how much time remains until the next bar begins.

However, rarely, if ever, will your first bar be exactly one minute long. Another issue is that you will hardly ever manage to start the application at the precise zero-second mark of the system clock. But in this case, we might implement a minor adjustment or rather some test that forces the replay/simulator to start in a way that creates synchronization with the system clock. This solution is viable and plausible. Still, if you signal that the application is ready and a bar can begin to be plotted, you would then need to wait for a certain amount of time before synchronization could actually occur. That waiting time wouldn't be particularly long: at most, 59 seconds in the worst-case scenario. This is acceptable for a personal-use application. But even in a personal-use context, sooner or later you get tired of waiting up to 59 seconds every time you hit play on the application.

Even though forcing the replay/simulator to stay synchronized with the system clock simplifies our job of tracking the bar's remaining time, it makes the user experience somewhat irritating. The worst case is when you pause the simulation or replay in the middle. In that case, you'd need to wait up to 59 seconds again before the application could resume chart plotting, because it would have to wait for the system clock to align once more. So, practically speaking, this doesn't seem like the best solution.

However, we might be able to find a middle ground. Something that lets us perform a slight adjustment to stay in sync with the system clock, while avoiding the need to wait for up to 59 seconds for the system clock to "unlock" bar plotting. That way, we can still determine the time remaining in the bar. Now things start to get a bit more interesting.

I'm providing this detailed explanation not to make things overly complicated or to confuse you, dear reader, but to show that before we sit down to write code, we need to think first. We must analyze the possibilities and weigh the implementation difficulty. Many people believe that programming is just about throwing a bunch of code into a file and calling it done. But in truth, coding is only a small part of the process. The most important part lies in the planning and analysis of the solution. That's what we've done up until now. With this thought process, we can form a solid idea of what needs to be done. However, some changes will still be necessary. We will need to improve our organization a bit. But to truly understand what we're going to do, let's move on to a new section.

Implementing the Base Version for Displaying the Remaining Time

What we are going to do in this section is something that, in my view, is relatively simple but also quite bold. We will implement a way to quickly relay the current bar time information to the mouse indicator. We will begin by making some changes to the mouse indicator code. These changes will involve adding a bit more code, as shown in the snippet below:

12. //+------------------------------------------------------------------+
13. double GL_PriceClose;
14. datetime GL_TimeAdjust;
15. //+------------------------------------------------------------------+
16. #include <Market Replay\Auxiliar\Study\C_Study.mqh>
17. //+------------------------------------------------------------------+
18. C_Study *Study       = NULL;
19. //+------------------------------------------------------------------+
20. input color user02   = clrBlack;                         //Price Line
21. input color user03   = clrPaleGreen;                     //Positive Study
22. input color user04   = clrLightCoral;                    //Negative Study
23. //+------------------------------------------------------------------+
24. C_Study::eStatusMarket m_Status;
25. int m_posBuff = 0;
26. double m_Buff[];
27. //+------------------------------------------------------------------+
28. int OnInit()
29. {
30.    ResetLastError();
31.    Study = new C_Study(0, "Indicator Mouse Study", user02, user03, user04);
32.    if (_LastError != ERR_SUCCESS) return INIT_FAILED;
33.    if ((*Study).GetInfoTerminal().szSymbol != def_SymbolReplay)
34.    {
35.       MarketBookAdd((*Study).GetInfoTerminal().szSymbol);
36.       OnBookEvent((*Study).GetInfoTerminal().szSymbol);
37.       m_Status = C_Study::eCloseMarket;
38.    }else
39.       m_Status = C_Study::eInReplay;
40.    SetIndexBuffer(0, m_Buff, INDICATOR_DATA);
41.    ArrayInitialize(m_Buff, EMPTY_VALUE);
42.    
43.    return INIT_SUCCEEDED;
44. }
45. //+------------------------------------------------------------------+
46. int OnCalculate(const int rates_total, const int prev_calculated, const datetime& time[], const double& open[],
47.                 const double& high[], const double& low[], const double& close[], const long& tick_volume[], 
48.                 const long& volume[], const int& spread[]) 
49. {
50.    GL_PriceClose = close[rates_total - 1];
51.    GL_TimeAdjust = (spread[rates_total - 1] < 60 ? spread[rates_total - 1] : 0);
52.    m_posBuff = rates_total;
53.    (*Study).Update(m_Status);   
54.    
55.    return rates_total;
56. }
57. //+------------------------------------------------------------------+

Source code fragment of Mouse Study.mq5

If you compare the code snippet shown above with the latest source code of the mouse indicator, you will notice that line 14 has been added to the code. In other words, we now have a new global variable within the indicator module. I have no intention of adding new global variables to this mouse indicator module. But this one is special, for a reason you'll soon understand. In any case, this variable will be assigned a value on line 51. This value will be passed via the spread. Therein lies a potential risk which at the same time hints at something quite helpful. Regardless, this global variable is only used in a very specific way. To better understand this, let's take a look at the updated C_Study.mqh file, shown below:

001. //+------------------------------------------------------------------+
002. #property copyright "Daniel Jose"
003. //+------------------------------------------------------------------+
004. #include "..\C_Mouse.mqh"
005. //+------------------------------------------------------------------+
006. #define def_ExpansionPrefix def_MousePrefixName + "Expansion_"
007. //+------------------------------------------------------------------+
008. class C_Study : public C_Mouse
009. {
010.    private   :
011. //+------------------------------------------------------------------+
012.       struct st00
013.       {
014.          eStatusMarket  Status;
015.          MqlRates       Rate;
016.          string         szInfo,
017.                         szBtn1,
018.                         szBtn2,
019.                         szBtn3;
020.          color          corP,
021.                         corN;
022.          int            HeightText;
023.          bool           bvT, bvD, bvP;
024.          datetime       TimeDevice;
025.       }m_Info;
026. //+------------------------------------------------------------------+
027.       void Draw(void)
028.          {
029.             double v1;
030.             
031.             if (m_Info.bvT)
032.             {
033.                ObjectSetInteger(GetInfoTerminal().ID, m_Info.szBtn1, OBJPROP_YDISTANCE, GetInfoMouse().Position.Y_Adjusted - 18);
034.                ObjectSetString(GetInfoTerminal().ID, m_Info.szBtn1, OBJPROP_TEXT, m_Info.szInfo);
035.             }
036.             if (m_Info.bvD)
037.             {
038.                v1 = NormalizeDouble((((GetInfoMouse().Position.Price - m_Info.Rate.close) / m_Info.Rate.close) * 100.0), 2);
039.                ObjectSetInteger(GetInfoTerminal().ID, m_Info.szBtn2, OBJPROP_YDISTANCE, GetInfoMouse().Position.Y_Adjusted - 1);
040.                ObjectSetInteger(GetInfoTerminal().ID, m_Info.szBtn2, OBJPROP_BGCOLOR, (v1 < 0 ? m_Info.corN : m_Info.corP));
041.                ObjectSetString(GetInfoTerminal().ID, m_Info.szBtn2, OBJPROP_TEXT, StringFormat("%.2f%%", MathAbs(v1)));
042.             }
043.             if (m_Info.bvP)
044.             {
045.                v1 = NormalizeDouble((((GL_PriceClose - m_Info.Rate.close) / m_Info.Rate.close) * 100.0), 2);
046.                ObjectSetInteger(GetInfoTerminal().ID, m_Info.szBtn3, OBJPROP_YDISTANCE, GetInfoMouse().Position.Y_Adjusted - 1);
047.                ObjectSetInteger(GetInfoTerminal().ID, m_Info.szBtn3, OBJPROP_BGCOLOR, (v1 < 0 ? m_Info.corN : m_Info.corP));
048.                ObjectSetString(GetInfoTerminal().ID, m_Info.szBtn3, OBJPROP_TEXT, StringFormat("%.2f%%", MathAbs(v1)));
049.             }
050.          }
051. //+------------------------------------------------------------------+
052. inline void CreateObjInfo(EnumEvents arg)
053.          {
054.             switch (arg)
055.             {
056.                case evShowBarTime:
057.                   C_Mouse::CreateObjToStudy(2, 110, m_Info.szBtn1 = (def_ExpansionPrefix + (string)ObjectsTotal(0)), clrPaleTurquoise);
058.                   m_Info.bvT = true;
059.                   break;
060.                case evShowDailyVar:
061.                   C_Mouse::CreateObjToStudy(2, 53, m_Info.szBtn2 = (def_ExpansionPrefix + (string)ObjectsTotal(0)));
062.                   m_Info.bvD = true;
063.                   break;
064.                case evShowPriceVar:
065.                   C_Mouse::CreateObjToStudy(58, 53, m_Info.szBtn3 = (def_ExpansionPrefix + (string)ObjectsTotal(0)));
066.                   m_Info.bvP = true;
067.                   break;
068.             }
069.          }
070. //+------------------------------------------------------------------+
071. inline void RemoveObjInfo(EnumEvents arg)
072.          {
073.             string sz;
074.             
075.             switch (arg)
076.             {
077.                case evHideBarTime:
078.                   sz = m_Info.szBtn1;
079.                   m_Info.bvT = false;
080.                   break;
081.                case evHideDailyVar:
082.                   sz = m_Info.szBtn2;
083.                   m_Info.bvD   = false;
084.                   break;
085.                case evHidePriceVar:
086.                   sz = m_Info.szBtn3;
087.                   m_Info.bvP = false;
088.                   break;
089.             }
090.             ChartSetInteger(GetInfoTerminal().ID, CHART_EVENT_OBJECT_DELETE, false);
091.             ObjectDelete(GetInfoTerminal().ID, sz);
092.             ChartSetInteger(GetInfoTerminal().ID, CHART_EVENT_OBJECT_DELETE, true);
093.          }
094. //+------------------------------------------------------------------+
095.    public   :
096. //+------------------------------------------------------------------+
097.       C_Study(long IdParam, string szShortName, color corH, color corP, color corN)
098.          :C_Mouse(IdParam, szShortName, corH, corP, corN)
099.          {
100.             if (_LastError != ERR_SUCCESS) return;
101.             ZeroMemory(m_Info);
102.             m_Info.Status = eCloseMarket;
103.             m_Info.Rate.close = iClose(GetInfoTerminal().szSymbol, PERIOD_D1, ((GetInfoTerminal().szSymbol == def_SymbolReplay) || (macroGetDate(TimeCurrent()) != macroGetDate(iTime(GetInfoTerminal().szSymbol, PERIOD_D1, 0))) ? 0 : 1));
104.             m_Info.corP = corP;
105.             m_Info.corN = corN;
106.             CreateObjInfo(evShowBarTime);
107.             CreateObjInfo(evShowDailyVar);
108.             CreateObjInfo(evShowPriceVar);
109.          }
110. //+------------------------------------------------------------------+
111.       void Update(const eStatusMarket arg)
112.          {
113.             int i0;
114.             datetime dt;
115.                      
116.             switch (m_Info.Status = (m_Info.Status != arg ? arg : m_Info.Status))
117.             {
118.                case eCloseMarket   :
119.                   m_Info.szInfo = "Closed Market";
120.                   break;
121.                case eInReplay      :
122.                case eInTrading     :
123.                   i0 = PeriodSeconds();
124.                   dt = (m_Info.Status == eInReplay ? m_Info.TimeDevice + GL_TimeAdjust : TimeCurrent());
125.                   m_Info.Rate.time = (m_Info.Rate.time <= dt ? (datetime)(((ulong) dt / i0) * i0) + i0 : m_Info.Rate.time);
126.                   m_Info.szInfo = TimeToString((datetime)m_Info.Rate.time - dt, TIME_SECONDS);
127.                   break;
128.                case eAuction      :
129.                   m_Info.szInfo = "Auction";
130.                   break;
131.                default            :
132.                   m_Info.szInfo = "ERROR";
133.             }
134.             Draw();
135.          }
136. //+------------------------------------------------------------------+
137. virtual void DispatchMessage(const int id, const long &lparam, const double &dparam, const string &sparam)
138.          {
139.             C_Mouse::DispatchMessage(id, lparam, dparam, sparam);
140.             switch (id)
141.             {
142.                case CHARTEVENT_CUSTOM + evHideBarTime:
143.                   RemoveObjInfo(evHideBarTime);
144.                   break;
145.                case CHARTEVENT_CUSTOM + evShowBarTime:
146.                   CreateObjInfo(evShowBarTime);
147.                   break;
148.                case CHARTEVENT_CUSTOM + evHideDailyVar:
149.                   RemoveObjInfo(evHideDailyVar);
150.                   break;
151.                case CHARTEVENT_CUSTOM + evShowDailyVar:
152.                   CreateObjInfo(evShowDailyVar);
153.                   break;
154.                case CHARTEVENT_CUSTOM + evHidePriceVar:
155.                   RemoveObjInfo(evHidePriceVar);
156.                   break;
157.                case CHARTEVENT_CUSTOM + evShowPriceVar:
158.                   CreateObjInfo(evShowPriceVar);
159.                   break;
160.                case (CHARTEVENT_CUSTOM + evSetServerTime):
161.                   m_Info.TimeDevice = (datetime)lparam;
162.                   break;
163.                case CHARTEVENT_MOUSE_MOVE:
164.                   Draw();
165.                   break;
166.             }
167.             ChartRedraw(GetInfoTerminal().ID);
168.          }
169. //+------------------------------------------------------------------+
170. };
171. //+------------------------------------------------------------------+
172. #undef def_ExpansionPrefix
173. #undef def_MousePrefixName
174. //+------------------------------------------------------------------+

C_Study.mqh source code

Alright. Very good. Pay attention, because what is about to be explained is extremely important for understanding the overall functioning. You'll notice that the code in the header file looks different. However, there are only two truly important parts here. The first is on line 160, where we handle a custom event. Note that on line 161, we store a value passed by this event into a private variable within the class. That's the first point. Now, let's look at where the real "magic" happens. To do that, let's scroll up a bit to line 111, where the Update procedure is defined. This procedure is responsible for creating the information that we'll later display. Now pay attention. On line 123, we capture the number of seconds present within the current timeframe of the custom symbol chart. Then, on line 124, we perform a small calculation or, alternatively, we use the value provided by MetaTrader 5. Whether we perform the calculation or use the provided value depends on the status of the indicator. When the mouse indicator is being used on a replay symbol, we perform the calculation. Otherwise, we use the value supplied by MetaTrader 5.

Note that this calculation takes into account the value we captured from the mouse indicator code snippet, along with another value provided by the replay/simulator application. We will see later how this second value is passed to the mouse indicator.

In any case, what we have is a data point that will change over time. However, we also need another value, which is calculated just below, on line 125. This line determines the exact moment when a new bar will be generated on the chart. hen on line 126, we perform one final calculation to determine and present to the user the remaining time until the current bar closes.

This entire system will function properly because the entity responsible for telling us how much time remains until a new bar is formed is the trading server. Or, in the case of using replay/simulation, the entity responsible is the service that updates the bars, allowing MetaTrader 5 to render them.

Very good. You’ve probably noticed that we need to modify the service code due to the explanations above. However, what actually needs updating is the code in the C_Replay.mqh file. But before we do that, we need to make a small modification in two areas of the service code. The first is to add a few things to the Macros.mqh header file. Here's how to do it:

1. //+------------------------------------------------------------------+
2. #property copyright "Daniel Jose"
3. //+------------------------------------------------------------------+
4. #define macroRemoveSec(A) (A - (A % 60))
5. #define macroGetDate(A)   (A - (A % 86400))
6. #define macroGetSec(A)    (A - (A - (A % 60)))
7. //+------------------------------------------------------------------+

Macros.mqh source code

Since everything is very simple, I will not go into details. Once this is done, we will modify the C_FilesTick.mqh header file. The modification is of a special nature and is shown in the following fragment:

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. #include "C_FileBars.mqh"
05. #include "C_Simulation.mqh"
06. #include "..\..\Auxiliar\Macros.mqh"
07. //+------------------------------------------------------------------+
08. //#define macroRemoveSec(A)   (A - (A % 60))
09. #define def_MaxSizeArray    16777216 // 16 Mbytes
10. //+------------------------------------------------------------------+
11. class C_FileTicks
12. {
13.    protected:
14.       enum ePlotType {PRICE_EXCHANGE, PRICE_FOREX};

C_FilesTick.mqh source code snippet

Note that in this case line 06 was added, and line 08, which we highlighted, should be removed from the code. This happens because the Macros.mqh header file now contains the code that was previously on the highlighted line. Very good. After making these changes, we can start working with the C_Replay.mqh file. The new code for this file is presented below in full.

001. //+------------------------------------------------------------------+
002. #property copyright "Daniel Jose"
003. //+------------------------------------------------------------------+
004. #include "C_ConfigService.mqh"
005. #include "C_Controls.mqh"
006. //+------------------------------------------------------------------+
007. #define def_IndicatorControl   "Indicators\\Market Replay.ex5"
008. #resource "\\" + def_IndicatorControl
009. //+------------------------------------------------------------------+
010. #define def_CheckLoopService ((!_StopFlag) && (ChartSymbol(m_Infos.IdReplay) != ""))
011. //+------------------------------------------------------------------+
012. #define def_ShortNameIndControl    "Market Replay Control"
013. #define def_MaxSlider             (def_MaxPosSlider + 1)
014. //+------------------------------------------------------------------+
015. class C_Replay : public C_ConfigService
016. {
017.    private   :
018.       struct st00
019.       {
020.          C_Controls::eObjectControl Mode;
021.            uCast_Double             Memory;
022.          ushort                     Position;
023.          int                        Handle;
024.       }m_IndControl;
025.       struct st01
026.       {
027.          long     IdReplay;
028.          int      CountReplay;
029.          double   PointsPerTick;
030.          MqlTick  tick[1];
031.          MqlRates Rate[1];
032.       }m_Infos;
033.       stInfoTicks m_MemoryData;
034. //+------------------------------------------------------------------+
035. inline bool MsgError(string sz0) { Print(sz0); return false; }
036. //+------------------------------------------------------------------+
037. inline void UpdateIndicatorControl(void)
038.          {
039.             double Buff[];
040.                                  
041.             if (m_IndControl.Handle == INVALID_HANDLE) return;
042.             if (m_IndControl.Memory._16b[C_Controls::eCtrlPosition] == m_IndControl.Position)
043.             {
044.                if (CopyBuffer(m_IndControl.Handle, 0, 0, 1, Buff) == 1)
045.                   m_IndControl.Memory.dValue = Buff[0];
046.                if ((m_IndControl.Mode = (C_Controls::eObjectControl)m_IndControl.Memory._16b[C_Controls::eCtrlStatus]) == C_Controls::ePlay)
047.                   m_IndControl.Position = m_IndControl.Memory._16b[C_Controls::eCtrlPosition];
048.             }else
049.             {
050.                m_IndControl.Memory._16b[C_Controls::eCtrlPosition] = m_IndControl.Position;
051.                m_IndControl.Memory._16b[C_Controls::eCtrlStatus] = (ushort)m_IndControl.Mode;
052.                m_IndControl.Memory._8b[7] = 'D';
053.                m_IndControl.Memory._8b[6] = 'M';
054.                EventChartCustom(m_Infos.IdReplay, evCtrlReplayInit, 0, m_IndControl.Memory.dValue, "");
055.             }
056.          }
057. //+------------------------------------------------------------------+
058.       void SweepAndCloseChart(void)
059.          {
060.             long id;
061.             
062.             if ((id = ChartFirst()) > 0) do
063.             {
064.                if (ChartSymbol(id) == def_SymbolReplay)
065.                   ChartClose(id);
066.             }while ((id = ChartNext(id)) > 0);
067.          }
068. //+------------------------------------------------------------------+
069. inline void CreateBarInReplay(bool bViewTick)
070.          {
071.             bool    bNew;
072.             double dSpread;
073.             int    iRand = rand();
074. 
075.             if (BuildBar1Min(m_Infos.CountReplay, m_Infos.Rate[0], bNew))
076.             {
077.                m_Infos.tick[0] = m_MemoryData.Info[m_Infos.CountReplay];
078.                if (m_MemoryData.ModePlot == PRICE_EXCHANGE)
079.                {                  
080.                   dSpread = m_Infos.PointsPerTick + ((iRand > 29080) && (iRand < 32767) ? ((iRand & 1) == 1 ? m_Infos.PointsPerTick : 0 ) : 0 );
081.                   if (m_Infos.tick[0].last > m_Infos.tick[0].ask)
082.                   {
083.                      m_Infos.tick[0].ask = m_Infos.tick[0].last;
084.                      m_Infos.tick[0].bid = m_Infos.tick[0].last - dSpread;
085.                   }else if (m_Infos.tick[0].last < m_Infos.tick[0].bid)
086.                   {
087.                      m_Infos.tick[0].ask = m_Infos.tick[0].last + dSpread;
088.                      m_Infos.tick[0].bid = m_Infos.tick[0].last;
089.                   }
090.                }
091.                if (bViewTick)
092.                {
093.                   CustomTicksAdd(def_SymbolReplay, m_Infos.tick);
094.                   if (bNew) EventChartCustom(m_Infos.IdReplay, evSetServerTime, (long)m_Infos.Rate[0].time, 0, "");
095.                }
096.                m_Infos.Rate[0].spread = (int)macroGetSec(m_MemoryData.Info[m_Infos.CountReplay].time);
097.                CustomRatesUpdate(def_SymbolReplay, m_Infos.Rate);
098.             }
099.             m_Infos.CountReplay++;
100.          }
101. //+------------------------------------------------------------------+
102.       void AdjustViewDetails(void)
103.          {
104.             MqlRates rate[1];
105. 
106.             ChartSetInteger(m_Infos.IdReplay, CHART_SHOW_ASK_LINE, GetInfoTicks().ModePlot == PRICE_FOREX);
107.             ChartSetInteger(m_Infos.IdReplay, CHART_SHOW_BID_LINE, GetInfoTicks().ModePlot == PRICE_FOREX);
108.             ChartSetInteger(m_Infos.IdReplay, CHART_SHOW_LAST_LINE, GetInfoTicks().ModePlot == PRICE_EXCHANGE);
109.             m_Infos.PointsPerTick = SymbolInfoDouble(def_SymbolReplay, SYMBOL_TRADE_TICK_SIZE);
110.             CopyRates(def_SymbolReplay, PERIOD_M1, 0, 1, rate);
111.             if ((m_Infos.CountReplay == 0) && (GetInfoTicks().ModePlot == PRICE_EXCHANGE))
112.                for (; GetInfoTicks().Info[m_Infos.CountReplay].volume_real == 0; m_Infos.CountReplay++);
113.             if (rate[0].close > 0)
114.             {
115.                if (GetInfoTicks().ModePlot == PRICE_EXCHANGE)
116.                   m_Infos.tick[0].last = rate[0].close;
117.                else
118.                {
119.                   m_Infos.tick[0].bid = rate[0].close;
120.                   m_Infos.tick[0].ask = rate[0].close + (rate[0].spread * m_Infos.PointsPerTick);
121.                }               
122.                m_Infos.tick[0].time = rate[0].time;
123.                m_Infos.tick[0].time_msc = rate[0].time * 1000;
124.             }else
125.                m_Infos.tick[0] = GetInfoTicks().Info[m_Infos.CountReplay];
126.             CustomTicksAdd(def_SymbolReplay, m_Infos.tick);
127.          }
128. //+------------------------------------------------------------------+
129.       void AdjustPositionToReplay(void)
130.          {
131.             int nPos, nCount;
132.             
133.             if (m_IndControl.Position == (int)((m_Infos.CountReplay * def_MaxSlider) / m_MemoryData.nTicks)) return;
134.             nPos = (int)((m_MemoryData.nTicks * m_IndControl.Position) / def_MaxSlider);
135.             for (nCount = 0; m_MemoryData.Rate[nCount].spread < nPos; m_Infos.CountReplay = m_MemoryData.Rate[nCount++].spread);
136.             if (nCount > 0) CustomRatesUpdate(def_SymbolReplay, m_MemoryData.Rate, nCount - 1);
137.             while ((nPos > m_Infos.CountReplay) && def_CheckLoopService)
138.                CreateBarInReplay(false);
139.          }
140. //+------------------------------------------------------------------+
141.    public   :
142. //+------------------------------------------------------------------+
143.       C_Replay()
144.          :C_ConfigService()
145.          {
146.             Print("************** Market Replay Service **************");
147.             srand(GetTickCount());
148.             SymbolSelect(def_SymbolReplay, false);
149.             CustomSymbolDelete(def_SymbolReplay);
150.             CustomSymbolCreate(def_SymbolReplay, StringFormat("Custom\\%s", def_SymbolReplay));
151.             CustomSymbolSetDouble(def_SymbolReplay, SYMBOL_TRADE_TICK_SIZE, 0);
152.             CustomSymbolSetDouble(def_SymbolReplay, SYMBOL_TRADE_TICK_VALUE, 0);
153.             CustomSymbolSetDouble(def_SymbolReplay, SYMBOL_VOLUME_STEP, 0);
154.             CustomSymbolSetString(def_SymbolReplay, SYMBOL_DESCRIPTION, "Symbol for replay / simulation");
155.             CustomSymbolSetInteger(def_SymbolReplay, SYMBOL_DIGITS, 8);
156.             SymbolSelect(def_SymbolReplay, true);
157.             m_Infos.CountReplay = 0;
158.             m_IndControl.Handle = INVALID_HANDLE;
159.             m_IndControl.Mode = C_Controls::ePause;
160.             m_IndControl.Position = 0;
161.             m_IndControl.Memory._16b[C_Controls::eCtrlPosition] = C_Controls::eTriState;
162.          }
163. //+------------------------------------------------------------------+
164.       ~C_Replay()
165.          {
166.             SweepAndCloseChart();
167.             IndicatorRelease(m_IndControl.Handle);
168.             SymbolSelect(def_SymbolReplay, false);
169.             CustomSymbolDelete(def_SymbolReplay);
170.             Print("Finished replay service...");
171.          }
172. //+------------------------------------------------------------------+
173.       bool OpenChartReplay(const ENUM_TIMEFRAMES arg1, const string szNameTemplate)
174.          {
175.             if (SymbolInfoDouble(def_SymbolReplay, SYMBOL_TRADE_TICK_SIZE) == 0)
176.                return MsgError("Asset configuration is not complete, it remains to declare the size of the ticket.");
177.             if (SymbolInfoDouble(def_SymbolReplay, SYMBOL_TRADE_TICK_VALUE) == 0)
178.                return MsgError("Asset configuration is not complete, need to declare the ticket value.");
179.             if (SymbolInfoDouble(def_SymbolReplay, SYMBOL_VOLUME_STEP) == 0)
180.                return MsgError("Asset configuration not complete, need to declare the minimum volume.");
181.             SweepAndCloseChart();
182.             m_Infos.IdReplay = ChartOpen(def_SymbolReplay, arg1);
183.             if (!ChartApplyTemplate(m_Infos.IdReplay, szNameTemplate + ".tpl"))
184.                Print("Failed apply template: ", szNameTemplate, ".tpl Using template default.tpl");
185.             else
186.                Print("Apply template: ", szNameTemplate, ".tpl");
187. 
188.             return true;
189.          }
190. //+------------------------------------------------------------------+
191.       bool InitBaseControl(const ushort wait = 1000)
192.          {
193.             Print("Waiting for Mouse Indicator...");
194.             Sleep(wait);
195.             while ((def_CheckLoopService) && (ChartIndicatorGet(m_Infos.IdReplay, 0, "Indicator Mouse Study") == INVALID_HANDLE)) Sleep(200);
196.             if (def_CheckLoopService)
197.             {
198.                AdjustViewDetails();
199.                Print("Waiting for Control Indicator...");
200.                if ((m_IndControl.Handle = iCustom(ChartSymbol(m_Infos.IdReplay), ChartPeriod(m_Infos.IdReplay), "::" + def_IndicatorControl, m_Infos.IdReplay)) == INVALID_HANDLE) return false;
201.                ChartIndicatorAdd(m_Infos.IdReplay, 0, m_IndControl.Handle);
202.                UpdateIndicatorControl();
203.             }
204.             
205.             return def_CheckLoopService;
206.          }
207. //+------------------------------------------------------------------+
208.       bool LoopEventOnTime(void)
209.          {         
210.             int iPos;
211. 
212.             while ((def_CheckLoopService) && (m_IndControl.Mode != C_Controls::ePlay))
213.             {
214.                UpdateIndicatorControl();
215.                Sleep(200);
216.             }
217.             m_MemoryData = GetInfoTicks();
218.             AdjustPositionToReplay();
219.             EventChartCustom(m_Infos.IdReplay, evSetServerTime, (long)macroRemoveSec(m_MemoryData.Info[m_Infos.CountReplay].time), 0, "");
220.             iPos = 0;
221.             while ((m_Infos.CountReplay < m_MemoryData.nTicks) && (def_CheckLoopService))
222.             {
223.                if (m_IndControl.Mode == C_Controls::ePause) return true;
224.                iPos += (int)(m_Infos.CountReplay < (m_MemoryData.nTicks - 1) ? m_MemoryData.Info[m_Infos.CountReplay + 1].time_msc - m_MemoryData.Info[m_Infos.CountReplay].time_msc : 0);
225.                CreateBarInReplay(true);
226.                while ((iPos > 200) && (def_CheckLoopService))
227.                {
228.                   Sleep(195);
229.                   iPos -= 200;
230.                   m_IndControl.Position = (ushort)((m_Infos.CountReplay * def_MaxSlider) / m_MemoryData.nTicks);
231.                   UpdateIndicatorControl();
232.                }
233.             }
234. 
235.             return ((m_Infos.CountReplay == m_MemoryData.nTicks) && (def_CheckLoopService));
236.          }
237. };
238. //+------------------------------------------------------------------+
239. #undef def_SymbolReplay
240. #undef def_CheckLoopService
241. #undef def_MaxSlider
242. //+------------------------------------------------------------------+

C_Replay.mqh source code

The problems here are a bit more complex than you might initially think. The reason is that we need to inform the mouse indicator directly about the exact point we're at in the simulation or replay. In a way, this wouldn't really be a problem. Earlier in this article I demonstrated how we could achieve that. However, you also noticed something strange was happening, and it became clear that we needed a different method to handle it properly.

That method was achieved by introducing three new lines of code. But don't be fooled into thinking this is a perfect solution. Because it's not. It merely solves one aspect of our problem. There is another aspect that this approach can't address, at least not yet. But before we dive into the limitations of this method, let's look at what those lines are doing.

We start with the simplest one, which is executed almost directly: line 96. If you paid attention to the mouse indicator source code fragment, you saw that we're using the spread value to achieve a fine-tuned adjustment in terms of seconds. Well, if we can't pass this value directly through the Rates, we'll send it another way as fast as possible. This is the method I found, given that throughout the simulation or replay we haven't been using the spread value for anything. So we'll use it in a more interesting way. There is a small problem here that does not concern us yet. For now we can live with this.

Continuing with the explanation of the new lines: if you look a little above, at line 94, you'll notice that each time the replay/simulation service detects a new bar, we trigger a custom event to notify the mouse indicator of the new value to use. Likewise, on line 219, we again tell the indicator which value to use. In both cases, this is done via a custom event.

But why do it this way? Isn't there another method to achieve the same result? Yes, there is. However, it didn't prove sufficiently adequate. At least not for what we still need to address. The fact is that these custom events are only triggered when we know a new one-minute bar has been closed. We could do this differently, without firing a custom event to the chart. For instance, we could use the iTime library function to determine when a new one-minute bar was created. Pay close attention to this detail: we don't care about the bar time in the current chart's timeframe; the timestamp of the one-minute bar matters. You might be a bit confused, but take a look at the code snippet below.

110. //+------------------------------------------------------------------+
111.       void Update(const eStatusMarket arg)
112.          {
113.             int i0;
114.             datetime dt;
115.                      
116.             switch (m_Info.Status = (m_Info.Status != arg ? arg : m_Info.Status))
117.             {
118.                case eCloseMarket   :
119.                   m_Info.szInfo = "Closed Market";
120.                   break;
121.                case eInReplay      :
122.                case eInTrading   :
123.                   i0 = PeriodSeconds();
124.                   dt = (m_Info.Status == eInReplay ? iTime(NULL, 0, 0) + GL_TimeAdjust : TimeCurrent());
125.                   m_Info.Rate.time = (m_Info.Rate.time <= dt ? (datetime)(((ulong) dt / i0) * i0) + i0 : m_Info.Rate.time);
126.                   m_Info.szInfo = TimeToString((datetime)m_Info.Rate.time - dt, TIME_SECONDS);
127.                   break;
128.                case eAuction      :
129.                   m_Info.szInfo = "Auction";
130.                   break;
131.                default            :
132.                   m_Info.szInfo = "ERROR";
133.             }
134.             Draw();
135.          }
136. //+------------------------------------------------------------------+

Code snippet from file C_Study.mqh

The change is in line 124. The use of this function to determine the bar time is a waste of time, since the resulting value will be the same as the one in the OnCalculate function. However, if you change the same fragment as shown below, everything will be completely different.

110. //+------------------------------------------------------------------+
111.       void Update(const eStatusMarket arg)
112.          {
113.             int i0;
114.             datetime dt;
115.                      
116.             switch (m_Info.Status = (m_Info.Status != arg ? arg : m_Info.Status))
117.             {
118.                case eCloseMarket   :
119.                   m_Info.szInfo = "Closed Market";
120.                   break;
121.                case eInReplay      :
122.                case eInTrading   :
123.                   i0 = PeriodSeconds();
124.                   dt = (m_Info.Status == eInReplay ? iTime(NULL, PERIOD_M1, 0) + GL_TimeAdjust : TimeCurrent());
125.                   m_Info.Rate.time = (m_Info.Rate.time <= dt ? (datetime)(((ulong) dt / i0) * i0) + i0 : m_Info.Rate.time);
126.                   m_Info.szInfo = TimeToString((datetime)m_Info.Rate.time - dt, TIME_SECONDS);
127.                   break;
128.                case eAuction      :
129.                   m_Info.szInfo = "Auction";
130.                   break;
131.                default            :
132.                   m_Info.szInfo = "ERROR";
133.             }
134.             Draw();
135.          }
136. //+------------------------------------------------------------------+

Code snippet from file C_Study.mqh

Note that the change lies solely in the fact that we are now asking MetaTrader 5 to tell us when the one-minute bar started. And simply doing this frees us from having to make the service trigger a custom event to pass us that same information. But wait a moment, I didn't quite get that! Well, my dear reader, the point is this: regardless of the chart's timeframe, the information MetaTrader 5 provides is precisely the timestamp of when the one-minute bar started. And that's exactly what the custom event is actually communicating. However, by handling this within the service, I ensure that the information is passed only at appropriate intervals rather than calling it from within the indicator and potentially overloading the system by requesting the same information repeatedly.

Final considerations

Although what we've seen here doesn't yet represent a definitive solution to the problem, it has proven to be quite suitable for the current stage. Therefore, I see no reason not to use it for now. Still, it's important to recognize that we'll soon need a more refined and robust solution. But until then, we now have a reliable way to detect when a new bar will be generated.

In the video below, you can see the current system in action.

Running a demo version