Working with real tick arrays in MqlTick structures

MetaTrader 5 provides the ability to work not only with the history of quotes (bars) but also with the history of real ticks. From the user interface, all historical data is available in the Symbols dialog. It has three tabs: Specification, Bars, and Ticks. When a specific element is selected in the tree-like list of symbols on the first tab, then when switching to tabs Bars and Ticks you can request quotes in the form of bars or ticks, respectively.

From MQL programs, the history of real ticks is also available using the CopyTicks and CopyTicksRange functions.

int CopyTicks(const string symbol, MqlTick &ticks[], uint flags = COPY_TICKS_ALL, ulong from = 0, uint count = 0)

int CopyTicksRange(const string symbol, MqlTick &ticks[], uint flags = COPY_TICKS_ALL, ulong from = 0, ulong to = 0)

Both functions request ticks for the specified instrument symbol into the array ticks passed by reference. Structure MqlTick contains all information about one tick and is described in MQL5 as follows:

The flags field is intended for storing a bit mask of signs, which fields in the tick structure contain changed values.

This was required because every tick always fills in all fields, regardless of whether the data has changed compared to the previous tick. This allows you to always have the current state of prices at any time without looking for previous values in the tick history. For example, only the Bid price could change with a tick, but in addition to the new price, other parameters will be indicated in the structure: previous Ask, Last, volume and so on.

At the same time, you should keep in mind that, depending on the type of instrument, some fields in ticks can always be zero (and the corresponding mask bits are never set for them). In particular, for Forex instruments, as a rule, the last, volume, volume_real fields remain empty.

The receiving ticks array can be of fixed size or dynamic. The functions will copy no more ticks into a fixed array than the size of the array, regardless of the actual number of ticks in the requested time interval (specified by the from/to parameters in the CopyTicksRange function) or in the count parameter of the CopyTicks function. In the ticks array, the oldest ticks are placed first, and the newest ticks are placed last.

In the parameters of both functions, time readings are specified as milliseconds since 01.01.1970 00:00:00. In the CopyTicks function, the range of requested ticks is set by the initial from and the number of ticks count, and in CopyTicksRange it is set by from and to (both values are included).

In other words, CopyTicksRange is designed to receive ticks in a specific interval, and their number is not known in advance. CopyTicks guarantees no more than count ticks but does not allow you to determine in advance what time interval these ticks will cover.

Chronological order of from and to values in CopyTicksRange is not important: the function will give ticks in any case, starting from the minimum of the two values, and ending with the maximum.

The CopyTicks function evaluates the from parameter as the left border with the minimum time and counts from it count ticks to the future. However, there is an important exception: from = 0 (by default) is treated as the current moment in time, and ticks are counted from it into the past. This makes it possible to always get the specified number of last ticks. When count = 0 (by default), the function copies no more than 2000 ticks.

Both functions return the number of copied ticks or -1 in case of an error. In particular, GetLastError may return the following error codes:

• ERR_HISTORY_TIMEOUT — tick synchronization timeout has expired, the function returned everything it had.
• ERR_HISTORY_SMALL_BUFFER — the static buffer is too small, so it gives as much as fits in the array.
• ERR_NOT_ENOUGH_MEMORY — failed to allocate the required amount of memory to get the history of ticks from the specified range into a dynamic array.

The flags parameter defines the type of ticks requested.

For any request types, the remaining fields of the MqlTick structure, which do not match the flags, will contain the previous actual values. For example, if only information ticks (COPY_TICKS_INFO) were requested, the remaining fields will still be filled in them. It means that if only the Bid price has changed, the last known values will be written in the ask and volume fields. To find out what has changed in the tick, analyze its flags field (there will be either the value TICK_FLAG_BID, or TICK_FLAG_ASK, or a combination of both). If a tick has zero values of the Bid and Ask prices, and the flags indicate that these prices have changed (flags == TICK_FLAG_BID | TICK_FLAG_ASK), then this indicates the emptying of the order book.

Similarly, if trading ticks were requested (COPY_TICKS_TRADE), the last known price values will be recorded in their bid and ask fields. In this case, the flags field may have a combination of TICK_FLAG_LAST, TICK_FLAG_VOLUME, TICK_FLAG_BUY, TICK_FLAG_SELL.

When requesting COPY_TICKS_ALL, all ticks are returned.

Calling any of the CopyTicks/CopyTicksRange functions checks the synchronization of the tick base stored on the hard disk for the given symbol. If there are not enough ticks in the local database, then the missing ticks will be automatically downloaded from the trade server. In this case, ticks will be synchronized taking into account the oldest date from the query parameters and up to the current moment. After that, all incoming ticks for this symbol will go to the tick database and keep it up to date in a synchronized state.

Tick data is much larger than minute quotes. When you first request a tick history or start testing by real ticks, downloading them can take a long time. The history of tick data is stored in files in the internal TKC format in the directory {terminal_dir}/bases/{server_name}/ticks/{symbol_name}. Each file contains information for one month.

In indicators, the functions return the result immediately, that is, they copy the available ticks by symbol and start the background process of tick base synchronization if there is not enough data. All indicators on one symbol work in one common thread, so they don't have a right to wait for the synchronization to complete. After the end of synchronization, the next call of the function will return all the requested ticks.

In Expert Advisors and scripts, functions can wait for up to 45 seconds for a result: unlike an indicator, each Expert Advisor and script runs in its own thread and therefore can wait for synchronization to complete within a timeout. If during this time ticks are still not synchronized in the required amount, then only available ticks will be returned, and synchronization will continue in the background.

Recall that real-time ticks are broadcast to charts as events: indicators receive notifications of new ticks in the OnCalculate handler, while Expert Advisors receive them in the OnTick handler. It should be borne in mind that the system does not guarantee the delivery of all events. If new ticks arrive in the terminal while the program is processing the current OnCalculate/OnTick event, new events for this "busy" program may not be added to its queue (see section Overview of event handling functions). Moreover, several ticks can arrive at the same time, but only one event will be generated for each MQL program: the current market state event. In this case, you can use the CopyTicks function to request all ticks that have come since the previous processing of the event. Here is what this algorithm looks like in pseudocode:

The SymbolInfoTick function used here populates a single MqlTick structure passed by reference with the last tick data. We will study it in a separate section.

Note that when calling CopyTicks, one millisecond is added to the old timestamp prev. This ensures that the previous tick is not processed again. However, if there were several ticks within one millisecond corresponding to prev, this algorithm will skip them. If you want to cover absolutely all ticks, you should remember the number of available ticks with the prev time while updating the prev variable. On the next CopyTicks call, query ticks from the prev moment and skip (ignore in the array) the number of "old" ticks.

However, please note that the above algorithm is not required by every MQL program. Most of them do not analyze each tick, while the current price state corresponding to the last known tick is quickly broadcast to charts in the events model and is available through symbol and chart properties.

To demonstrate the functions, let's consider two examples, one for each function. For both examples, a common header file TickEnum.mqh was developed, where the above constants for requested tick flags and tick status flags are summarized into two enumerations.

The use of enumerations makes type checking in source code more rigorous, and it also makes it easier to display the meaning of values as strings with EnumToString. In addition, the most popular combinations of flags have been added to the TICK_FLAGS enumeration to optimize the visualization or filtering of ticks. It is not possible to give enumeration elements the same names as built-in constants, as a name conflict occurs.

The first script SeriesTicksStats.mq5 uses the CopyTicks function to count the number of ticks with different flags set to a given history depth.

In the input parameters, you can set the working symbol (chart symbol by default), the number of analyzed ticks, and the request mode from COPY_TICKS.

The statistics of the occurrence of each flag (each bit in the bit mask) in the tick properties are collected in the TickFlagStats structure.

The OnStart function describes an array of TickFlagStats structures with a size of 8 elements: 6 of them (from 1 to 6 inclusive) are used for the corresponding TICK_FLAG bits, and the other two are used for bit combinations (see below). Using a simple loop, elements for individual standard bits/flags are filled in the array, and after the loop, two combined masks are filled (in the 0th element, ticks will be counted with a simultaneous change of Bid and Ask, and in the 7th element we count ticks with simultaneous Buy and Sell deals).

We will entrust all the main work to the auxiliary function CalcTickStats, passing input parameters and a prepared array for collecting statistics to it. After that, it remains to display the counted numbers in the journal.

The CalcTickStats function itself is very interesting.

It uses CopyTicks to request ticks of the specified symbol, of a specific type, starting from the start date, in the amount of count items. The start parameter is of the type datetime, and it must be converted to milliseconds when passed to CopyTicks. Recall that if start = 0 (which is the case here, in the OnStart function), the system will return the last ticks, counting from the current time. Therefore, each time the script is called, the statistics will most likely be updated due to the arrival of new ticks. The only possible exceptions are requests on weekends or those for low-liquid instruments.

If CopyTicks executes without errors, our code logs the time range covered by the received ticks.

Finally, in the loop, we go through all the ticks and count the number of bitwise matches in the tick flags and element masks in the array of statistical structures TickFlagStats prepared in advance.

It is advisable to run the script on instruments where there is information about real volumes and deals in order to test all modes from the COPY_TICKS enumeration (remember, they correspond to the constants for the flags parameter in CopyTicks: COPY_TICKS_INFO, COPY_TICKS_TRADE and COPY_TICKS_ALL).

Here is an example of log entries when requesting statistics for 100000 ticks of all types (TickType = ALL_TICKS):

Here is what you get when requesting only information ticks (TickType = INFO_TICKS).

Here you can check the accuracy of the calculations: the sum of the numbers for TF_BID and TF_ASK minus the matches TF_BID_ASK (COMBO) gives exactly 100000 (total number of ticks). Ticks with volumes and Last prices did not get into the result, as it was expected.

Now let's run the script again, exclusively for trading ticks (TickType = TRADE_TICKS).

All ticks had TF_LAST and TF_VOLUME flags, and trade direction mixing happened 7308 times. Again, the sum of TF_BUY and TF_SELL minus their combination coincides with the total number of ticks.

The second script SeriesTicksDeltaVolume.mq5 uses the CopyTicksRange function to calculate the volume deltas on each bar. As you know, MetaTrader 5 quotes contain only impersonal volumes, in which purchases and sales are combined in one value for each bar. However, the presence of a history of real ticks allows you to calculate separately the sums of buy and sell volumes, as well as their difference. These characteristics are additional important factors for making trading decisions.

The input parameters contain similar settings as in the first script, in particular, the symbol name for analysis, and the tick request mode. True, in this case, you will additionally need to specify a timeframe, because volume deltas should be calculated bar by bar. The current chart timeframe will be used by default. The BarCount parameter is used to specify the number of calculated bars.

Statistics for each bar are stored in the DeltaVolumePerBar structure.

The OnStart function describes an array of such structures, while its size is allocated for the specified number of bars.

And here is the main algorithm.

In the loop through bars, we get the time range for each bar: prev and next (0th incomplete bar is not processed). When calling CopyTicksRange for this interval, remember to translate datetime into milliseconds and subtract 1 millisecond from the right border, since this time belongs to the next bar. In the absence of errors, we process the array of received ticks in a loop.

If analysis by trading ticks (TRADE_TICKS) was requested in the script settings, check the presence of the TICK_FLAG_BUY and TICK_FLAG_SELL flags, and if at least one of them is set, take into account the volume from the volume field in the corresponding variable of the DeltaVolumePerBar structure. This mode is suitable only for stock instruments. For Forex instruments, volumes and trade direction flags are not filled, and therefore a different approach should be used.

If information ticks (INFO_TICKS) available for all instruments are specified in the settings, the algorithm is based on the following empirical rules. As you know, buying pushes the price up, and selling pushes it down. Therefore, we can assume that if the average price Ask+Bid moved up in a new tick relative to the previous one, a buy operation was executed on it, and if the price moved down, there was a sell operation. Volume can be roughly estimated as the number of points passed (_Point).

The calculation results are displayed simply as an array of structures with collected statistics.

Below are some logs for the TRADE_TICKS and INFO_TICKS modes.

The values, of course, are significantly different, but the point is not in absolute values: in the absence of exchange volumes, even such an emulation of splitting and delta dynamics allows us to look at the market behavior from a different angle.

When we learn how to create indicators, we will be able to embed this algorithm into one of them (see IndDeltaVolume.mq5 in the section Waiting for data and managing visibility) to visually display deltas directly on the chart.