Feature Engineering for ML (Part 4): Implementing Time Features in MQL5

Table of Contents

1. Introduction
2. The UTC Problem in MQL5
3. Architecture: CTimeFeatures and CRingBuffer
4. Cyclical Encoding in MQL5
5. Session Detection: The Cross-Midnight Edge Case
6. Session Volatility: The Ring Buffer
7. The Frequency Gate
8. EA Integration
9. Feature Count Reference
10. Conclusion
11. References

Introduction

Part 3 of this series built a Python function, get_time_features, that converts a DatetimeIndex into a structured feature matrix: Fourier-encoded cyclical variables for hour, day-of-week, and day-of-year; session membership flags for the four major forex trading sessions; a session overlap indicator; session-conditional rolling volatility; and calendar effects at period boundaries. The Python implementation operates on batch data with full vectorization. MQL5 operates bar by bar in real time.

This article implements the same feature set in MQL5 as a reusable include file, CTimeFeatures.mqh. Any EA can include it without modification. Three problems require solutions beyond the mechanical code translation. First, MQL5 provides broker time, not UTC — and most retail brokers offset their server clocks by two to three hours, which silently corrupts session boundary detection for every bar in the backtest. Second, Python computes rolling volatility in batch using pandas. In MQL5, volatility must be maintained incrementally per session using a circular buffer. Third, Python applies a frequency gate via a DataFrame.drop call. In MQL5, the gate must be set at initialization and mirrored in the feature-name registry used by downstream ONNX models.

The resulting class exposes three EA-facing methods — Initialize, Update, and Calculate — and produces a flat double array whose feature names and column order are identical to those of get_time_features. A model trained in Python on the Python features can be deployed in MQL5 via OnnxRun by passing the array from Calculate without any reordering.

The UTC Problem in MQL5

The single most consequential difference between the Python and MQL5 implementations is the time reference. Python's DatetimeIndex carries an explicit timezone. Calling tz_convert("UTC") on any timezone-aware index produces a correct UTC reference before applying session boundaries. MQL5 has no equivalent: iTime(), TimeCurrent(), and all bar timestamp functions return the broker's server time, which is whatever offset the broker has configured.

The dominant configuration among retail MetaTrader 5 brokers is UTC+2 in winter (EET) and UTC+3 in summer after the European DST transition. Applied to the London session (UTC 07:00–15:59), a UTC+2 offset shifts the window to 09:00–17:59 in broker time. An EA checking dt.hour >= 7 && dt.hour < 16 against broker time would classify every bar from 09:00 to 15:59 as London rather than 07:00 to 15:59 — a two-hour error that systematically misclassifies the London open and the Tokyo–London overlap for the entire backtest history.

Figure 1. A 2-panel illustration of session misclassification when broker time replaces UTC

• Panel (a): A UTC+2 broker shifts every session band two hours to the right. The green bars show correct UTC boundaries; the red bars show where those boundaries appear in broker time. The London open moves from 07:00 to 09:00 and the London–New York overlap window shifts by the same amount.
• Panel (b): A UTC+3 broker (EET summer, post-DST) produces a three-hour shift. A model trained on Python features derived from UTC data would receive mismatched session flags from an EA using broker time.

The fix is a single integer offset computed at initialization:

//--- Capture broker-to-UTC offset once at OnInit() time
m_gmt_offset = (int)(TimeGMT() - TimeCurrent());

TimeGMT() returns the current time in UTC. TimeCurrent() returns the last known server tick time in broker local time. Their difference is the signed offset in seconds that must be added to any broker timestamp to produce UTC. For a UTC+2 broker, m_gmt_offset evaluates to −7200 (minus two hours); for a UTC broker it is zero.

This offset is captured once at OnInit() time and held fixed for the life of the EA. The limitation is that it cannot track DST transitions that occur mid-session: if the broker advances its clock by one hour between the time OnInit() runs and a session boundary, the stored offset will be off by one hour for the remainder of that session. This is an acceptable approximation for features whose purpose is to label statistical session regimes, not to trigger at exact institutional session opens. Users who require minute-level session accuracy (for example, capturing the 07:00 UTC London open spike) should use a UTC-configured broker. This eliminates the problem entirely.

Architecture: CTimeFeatures and CRingBuffer

The implementation consists of two classes contained in a single header, CTimeFeatures.mqh.

CRingBuffer is a fixed-capacity circular buffer for scalar values. It supports Push to append a value and Std to compute the population standard deviation of all values currently in the buffer. Population standard deviation (ddof=0) is used rather than sample standard deviation to match NumPy's default, which is what Python's ring buffer simulation uses when reproducing the rolling volatility calculation. Each session and calendar column owns one CRingBuffer instance; nine buffers are maintained in total.

CTimeFeatures holds the configuration, the nine ring buffers, and the feature-name registry. It exposes three public methods to the EA:

• Initialize(timeframe, n_terms, forex, is_time_bar): Captures the UTC offset, sets frequency-gate flags, sizes and resets all ring buffers, and builds the feature-name array. When is_time_bar is false, Initialize adds two columns — bar_duration and bar_duration_accel — to the front of the feature registry; these columns are absent when is_time_bar is true. Must be called once from OnInit().
• Update(close_price, bar_broker_time): Computes the log return for the closed bar, determines which session and calendar volatility buffers are active at that bar's UTC time, and pushes the return into each active buffer. When is_time_bar is false, Update also computes the elapsed seconds since the previous bar and stores the result for the next Calculate call. Must be called on every closed bar before the corresponding Calculate call.
• Calculate(bar_broker_time, features[]): Populates the output array with all cyclical, session, calendar, and session-volatility features for the given bar time. Returns false if array allocation fails.

When is_time_bar=false, Update tracks the broker timestamp of each closed bar. On the second and subsequent calls it computes bar_duration as the elapsed seconds between the current bar's timestamp and the previous bar's timestamp, and bar_duration_accel as the first difference of that interval — the same arithmetic as Python's df.index.diff().dt.total_seconds() and its subsequent .diff(). Both values are stored in Update and read by Calculate on the next bar. This preserves the one-bar lag used for all features. For standard OHLCV time bars, pass is_time_bar=true; the two duration columns are absent from the output and do not appear in FeatureNames().

Figure 2. Architecture diagram of CTimeFeatures and its EA integration pattern

• Left box: The Expert Advisor's interface — four standard MQL5 event handlers. OnInit calls Initialize and then a warmup loop feeding historical bars to Update. OnTick calls Update on bar close and Calculate on each new bar open.
• Center box: The five public-facing methods of CTimeFeatures. The three functional methods delegate to three internal subsystems.
• Right column: The three internal subsystems — the cyclical encoder (computes sin/cos harmonics), the session detector (maps UTC hour to boolean flags), and the nine ring buffers (maintain rolling log-return history per session and calendar column).
• Bottom bar: The output array at H1/forex with n_terms=3: six cyclical features, five session flags, and nine session-volatility columns, totaling 20 features.

Cyclical Encoding in MQL5

The Fourier encoding in MQL5 reproduces the same arithmetic as Python with two coordinate conversions that are required to make the feature values identical.

Day-of-week alignment.MQL5's MqlDateTime.day_of_week runs 0=Sunday through 6=Saturday. Python's pd.DatetimeIndex.dayofweek runs 0=Monday through 6=Sunday. Feeding the MQL5 integer directly into the Fourier formula would encode Sunday at position 0 and Saturday at position 6, placing them at the same angular distance from Monday as they are from Friday — the wrong geometric relationship for a weekly cycle whose Monday represents the first active trading day. The conversion is:

//--- MQL5 day_of_week: 0=Sunday, 1=Monday ... 6=Saturday
//--- Python dayofweek: 0=Monday ... 6=Sunday
int py_dow = (dt.day_of_week == 0) ? 6 : dt.day_of_week - 1;

Day-of-year alignment. MQL5's MqlDateTime.day_of_year is zero-indexed: January 1 maps to 0. Python's DatetimeIndex.dayofyear is one-indexed: January 1 maps to 1. The cycle length in both cases is 366 (accommodating leap years). Adding 1 to the MQL5 value before the Fourier computation eliminates the offset:

int py_doy = dt.day_of_year + 1;   // MQL5: 0-based; Python: 1-based

With these two adjustments in place, the Fourier computation itself is identical in both languages:

//--- Hour: extra harmonics for sub-hourly bars; base harmonic only for H1+
int hour_harms = m_extra_hour_harms ? m_n_terms : 1;
for(int k = 1; k <= hour_harms; k++)
  {
   double ang = 2.0 * M_PI * k * hour / 24.0;
   features[idx++] = MathSin(ang);
   features[idx++] = MathCos(ang);
  }

//--- Day-of-week (base harmonic, cycle 7)
double dow_ang = 2.0 * M_PI * py_dow / 7.0;
features[idx++] = MathSin(dow_ang);
features[idx++] = MathCos(dow_ang);

//--- Day-of-year (base harmonic, cycle 366)
double doy_ang = 2.0 * M_PI * py_doy / 366.0;
features[idx++] = MathSin(doy_ang);
features[idx++] = MathCos(doy_ang);

The frequency gate variable m_extra_hour_harms is set to true when the timeframe is sub-hourly (M1 through M30). This replicates Python's behavior: for minute bars, the extra_fourier_features=["hour"] argument activates multiple harmonics for hour and appends a _h{k} suffix to each; for hourly and higher bars, extra_fourier_features=[] suppresses this and only the base harmonic (no suffix) is generated. The feature-name registry in _BuildNames mirrors this exactly, so FeatureNames() always returns the correct Python-compatible column names.

Session Detection: The Cross-Midnight Edge Case

Session membership is determined by the UTC hour extracted from the bar's converted timestamp. Three of the four sessions have contiguous windows that do not cross midnight: Tokyo (00:00–08:59), London (07:00–15:59), and New York (13:00–21:59). Their membership test is a simple AND condition. The Sydney session (21:00–05:59) straddles midnight and requires an OR condition — the same logic used in the Python implementation:

//+------------------------------------------------------------------+
//| InSession: true when UTC hour h falls inside session s           |
//+------------------------------------------------------------------+
bool CTimeFeatures::_InSession(const int h, const int s) const
  {
   if(m_s_xmid[s])
      return(h >= m_s_start[s] || h < m_s_end[s]);
   return(h >= m_s_start[s] && h < m_s_end[s]);
  }

Session boundaries are stored as instance arrays m_s_start[], m_s_end[], and m_s_xmid[] initialized in the constructor. The overlap indicator and the four calendar effect conditions are evaluated in _Activity, the private method that determines which of the nine ring buffers to push into for a given bar:

//+------------------------------------------------------------------+
//| Activity: populate the 9 active-buffer flags for a given bar     |
//+------------------------------------------------------------------+
void CTimeFeatures::_Activity(
   const int         hour,
   const MqlDateTime &dt,
   bool              &active[]
) const
  {
//--- MQL5 day_of_week: 0=Sunday; Python dayofweek: 0=Monday
   int py_dow = (dt.day_of_week == 0) ? 6 : dt.day_of_week - 1;

//--- Four main sessions
   int n_active = 0;
   for(int i = VB_SYDNEY; i <= VB_NY; i++)
     {
      active[i] = _InSession(hour, i);
      if(active[i])
         n_active++;
     }

//--- Overlap: two or more sessions simultaneously active
   active[VB_OVERLAP] = (n_active > 1);

//--- Calendar effects
//--- friday_ny_close: Python day_of_week==4 (Fri) and hour>=21
//--- sunday_open:     Python day_of_week==6 (Sun) and hour<=2
//--- month_end:       Python day >= 28
//--- quarter_end:     Python month%3==0 and day>=28
   active[VB_FRIDAY]      = (py_dow == 4 && hour >= 21);
   active[VB_SUNDAY_OPEN] = (dt.day_of_week == 0 && hour <= 2);
   active[VB_MONTH_END]   = (dt.day >= 28);
   active[VB_QUARTER_END] = (dt.mon % 3 == 0 && dt.day >= 28);
  }

One asymmetry in the calendar conditions is worth noting: sunday_open uses MQL5's native dt.day_of_week == 0 (Sunday in MQL5's encoding) because the condition tests for the first day of the week, not a day relative to Monday. The friday_ny_close condition uses the converted py_dow == 4 because it tests for the fifth business day, where the Python zero-Monday convention matters. Using the wrong convention for either condition would silently fire on the wrong day of the week.

Session Volatility: The Ring Buffer

In Python, session volatility is computed in a single pandas operation: mask the log-return series to within-session bars, call .rolling(20).std(ddof=0) (specifying ddof=0 to match NumPy's population standard deviation, which differs from pandas' default of ddof=1), forward-fill to all bars, and lag by one bar with .shift(1). This operation has access to the entire history at once. In MQL5, the equivalent must be maintained incrementally: each new closed bar provides one new log return, which is pushed into the ring buffer of every session that bar belongs to.

The CRingBuffer class holds a fixed-size array of double values in circular order. Push writes the new value at the current head position and advances the head, overwriting the oldest value once the buffer is full. Std iterates the occupied portion of the array to compute the population standard deviation:

   //+------------------------------------------------------------------+
   //| Push: append value at head position and advance the head index   |
   //+------------------------------------------------------------------+
   void CRingBuffer::Push(const double value)
     {
      m_data[m_head] = value;
      m_head         = (m_head + 1) % m_size;
      if(m_count < m_size)
         m_count++;
     }

   //+------------------------------------------------------------------+
   //| Std: population standard deviation (ddof=0) of buffered values   |
   //| Matches NumPy default used in Python's rolling volatility calc   |
   //+------------------------------------------------------------------+
   double CRingBuffer::Std(void) const
     {
      if(m_count < 2)
         return(0.0);
      double sum = 0.0, ssq = 0.0;
      for(int i = 0; i < m_count; i++)
        {
         sum += m_data[i];
         ssq += m_data[i] * m_data[i];
        }
      double mean = sum / m_count;
      double var  = ssq / m_count - mean * mean;
      return(var > 0.0 ? MathSqrt(var) : 0.0);
     }

The Update method of CTimeFeatures pushes the log return into each active buffer and immediately computes and stores the new standard deviation in m_last_vol[i]. For inactive sessions, m_last_vol[i] is left unchanged; this is the forward-fill that Python achieves with .reindex(method="ffill"). The one-bar lag is achieved structurally rather than explicitly: Update is called on the closed bar (bar index 1) before Calculate is called for the newly opened bar (bar index 0). When Calculate reads m_last_vol, the values it finds were last updated from bar 1's return, not bar 0's. No explicit shift(1) is required.

One deviation from Python is deliberate. Python computes volatility for all nine session and calendar columns. Therefore, the output includes friday_ny_close_vol, sunday_open_vol, month_end_vol, and quarter_end_vol even when the frequency gate drops the corresponding flag columns on sub-daily timeframes. This MQL5 implementation preserves that behavior: all nine volatility columns (the _vol-suffixed columns) appear in features[] regardless of timeframe. Dropping them would break ONNX model compatibility with models trained on the Python output.

Figure 3. 3-panel illustration of feature output over a 48 h M15 window (n_terms=3)

• Panel (a): Cyclical hour encoding with n_terms=3 on a sub-hourly timeframe. The k=1 pair (solid lines) encodes the dominant 24-hour cycle; the k=2 pair (dashed) adds a second oscillation per day, providing resolution for the bimodal intraday volatility structure. At H1 and higher, only k=1 is generated; k=2 and k=3 require both a sub-hourly timeframe and n_terms=3.
• Panel (b): Session flags for both trading days. Sydney (yellow) activates around 21:00 UTC and deactivates at 06:00. The Tokyo–London and London–New York overlaps (purple) are visible as the overlap indicator fires during the two daily overlap windows.
• Panel (c): Rolling volatility for the London session, New York session, and overlap. Each series updates only when its corresponding session is active and holds the previous estimate otherwise. The overlap series is elevated relative to individual sessions because it accumulates returns only from the highest-activity window of the day.

The Frequency Gate

The frequency gate controls which features are present in the output array depending on the timeframe passed to Initialize. It replicates the two-stage logic in Python's get_time_features:

1. Sub-hourly timeframes (M1–M30): The hour feature receives multiple harmonics up to n_terms with the _h{k} suffix. Day-of-week and day-of-year receive only the base harmonic with no suffix. Calendar effect flag columns are dropped.
2. Hourly timeframes (H1–H12): The hour feature receives only the base harmonic with no suffix. Day-of-week and day-of-year are the same as sub-hourly. Calendar effect flag columns are dropped.
3. Daily and higher timeframes (D1, W1, MN1): The hour feature receives only the base harmonic. The four calendar effect flag columns — friday_ny_close, sunday_open, month_end, quarter_end — are included in the output.

Two boolean flags, m_extra_hour_harms and m_include_calendar, are set in Initialize via two private helpers and checked in both Calculate and _BuildNames:

m_extra_hour_harms = _IsSubHourly();
m_include_calendar = _IsDailyPlus();

Both helpers use PeriodSeconds comparisons rather than explicit enum switches, which means the gate handles custom timeframes — PERIOD_H2, PERIOD_H3, PERIOD_H6 — without modification. Any period shorter than one hour activates extra hour harmonics; any period at least one day long activates calendar effects.

The feature-name array is built once at initialization, so FeatureCount() and FeatureNames() always reflect the actual column count for the configured timeframe. ONNX models that load a feature specification from file can call FeatureNames to validate at runtime that the live feature set matches the training-time specification:

string names[];
g_tf.FeatureNames(names);
if(ArraySize(names) != expected_feature_count)
  {
   Print("Feature count mismatch — check timeframe and n_terms");
   return(INIT_FAILED);
  }

EA Integration

Place CTimeFeatures.mqh in MQL5\Include\Features directory. Use angle-bracket syntax in all EA and script files: #include <CTimeFeatures.mqh>. Double-quote syntax is for same-directory relative includes and will fail when the EA compiles from MQL5\Experts\ or MQL5\Scripts\.

The canonical integration pattern is four lines in OnInit and four in OnTick. OnInit initializes the class, runs a warmup loop to seed the ring buffers with historical bar returns, and logs the feature count. OnTick detects new bars, calls Update on the just-closed bar, and calls Calculate on the newly opened bar:

//--- Global scope
#include <Features\CTimeFeatures.mqh>
CTimeFeatures g_tf;
input int InpNTerms = 3;    // Fourier harmonics
input int InpWarmup = 100;  // Historical bars for vol warmup

//+------------------------------------------------------------------+
//| OnInit: initialize CTimeFeatures and warm up ring buffers        |
//+------------------------------------------------------------------+
int OnInit(void)
  {
   if(!g_tf.Initialize(_Period, InpNTerms, true, true))
      return(INIT_FAILED);

   int total        = iBars(_Symbol, _Period);
   int warmup_start = MathMin(InpWarmup, total - 2);
   for(int i = warmup_start; i >= 1; i--)
      g_tf.Update(iClose(_Symbol, _Period, i),
                  iTime(_Symbol, _Period, i));

   PrintFormat("CTimeFeatures ready: %d features", g_tf.FeatureCount());
   return(INIT_SUCCEEDED);
  }


//+------------------------------------------------------------------+
//| OnTick: update buffers on bar close and compute features         |
//+------------------------------------------------------------------+
void OnTick(void)
  {
   static datetime s_last_bar = 0;
   datetime cur_bar = iTime(_Symbol, _Period, 0);
   if(cur_bar == s_last_bar)
      return;
   s_last_bar = cur_bar;

   //--- Update vol buffers with the just-closed bar (index 1)
   g_tf.Update(iClose(_Symbol, _Period, 1),
               iTime(_Symbol, _Period, 1));

   //--- Compute features for the newly opened bar (index 0)
   double features[];
   if(!g_tf.Calculate(cur_bar, features))
      return;

   //--- features[] is ready: pass to OnnxRun(), write to CSV, etc.
  }

Two details in the warmup loop require attention. The loop runs from warmup_start down to 1 — not to 0 — because bar 0 is the still-open current bar. Its close price is a moving target and should never enter a volatility buffer. The loop direction is oldest-to-newest (descending bar index) to preserve the chronological order that Update expects; reversing the loop direction would load returns in reverse time order, producing an incorrect ring buffer state.

The warmup depth of 100 bars is a practical minimum. CRingBuffer holds 20 values per buffer. Sydney session bars at H1 occur approximately 9 hours per day; in 100 H1 bars (roughly four trading days), the Sydney buffer accumulates around 37 entries — enough to reach full capacity. The London and New York buffers warm up faster. Calendar effect buffers — particularly quarter_end, which fires only on four occasions per year — will almost certainly not reach full capacity from a 100-bar warmup; their volatility estimates should be treated as unreliable until a full calendar quarter has been observed. For ONNX models that use all 20 features, providing at least 200–300 bars of warmup is more conservative.

Feature Count Reference

The table below gives the complete feature count for each timeframe class. All counts assume forex=true, is_time_bar=true, and the specified n_terms. The nine volatility columns (the _vol-suffixed columns) are always present regardless of timeframe because Python's frequency gate drops only the calendar flag columns, not their volatility counterparts. When is_time_bar=false, two additional columns — bar_duration and bar_duration_accel — are prepended before all other features, increasing every total by 2.

Timeframe	Cyclical	Flags	Volatility	Total (n_terms=3)
M1–M30	hour ×(2·n_terms) + dow×2 + doy×2	5	9	10 + 5 + 9 = 24
H1–H12	hour×2 + dow×2 + doy×2	5	9	6 + 5 + 9 = 20
D1, W1, MN1	hour×2 + dow×2 + doy×2	5 + 4 = 9	9	6 + 9 + 9 = 24

For verification, run TimeFeaturesVerify.mq5 as a script on EURUSD H1. It prints a tab-delimited header row followed by one row per bar to the Experts log. Copy those rows into a CSV file and compare them against the output of Python's get_time_features(df, timeframe="H1", forex=True) for the same date range. With a UTC broker, every cyclical and session flag value should match to six decimal places. The nine volatility columns in the script output will not match Python's shifted output exactly: the script calls Update immediately before Calculate for each printed bar, so the volatility values reflect the current bar's return rather than the prior bar's return. This is a deliberate simplification in the verification script to reduce its line count; the production integration pattern in Section 8 preserves the correct one-bar lag. With a UTC+2 broker, the session flag columns will additionally differ by the two-hour shift for all bars near session boundaries.

Conclusion

The three technical problems that separate the MQL5 implementation from a mechanical code translation each have a single correct solution. The UTC offset problem is resolved by capturing TimeGMT() − TimeCurrent() at initialization and adding it to every broker timestamp before session lookup; this eliminates the 2–3 hour classification error that would otherwise corrupt session flags for the entire backtest on any non-UTC broker. The incremental volatility problem is resolved by CRingBuffer: push on session-active bars, hold on inactive bars, and read the stored value in Calculate — a structure that replicates the pandas forward-fill and shift operations without access to the full history. The frequency gate problem is resolved by computing two boolean flags in Initialize and conditioning both Calculate and _BuildNames on them, so that FeatureCount() and the feature array always agree.

The output of Calculate is directly compatible with the Python output of get_time_features for the same timeframe and n_terms. An ONNX model trained on Python features for H1 EURUSD receives an array from OnnxRun that is numerically and structurally equivalent to what it was trained on; no reindexing, reordering, or feature renaming is required. Together with the fractionally differentiated price features from Part 2, this completes the MQL5 feature pipeline for the first two feature layers of the blueprint: price features and temporal context features.

References

1. López de Prado, M. (2018). Advances in financial machine learning. Wiley. Publisher link
2. Brockwell, P. J., & Davis, R. A. (2002). Introduction to time series and forecasting (2nd ed.). Springer. SpringerLink
3. Dacorogna, M., Gençay, R., Müller, U. A., Pictet, O. V., & Olsen, R. B. (2001). An introduction to high-frequency finance. Academic Press. ScienceDirect
4. MetaQuotes Ltd. (2024). MQL5 Reference: Date and Time Functions. MQL5 Reference Link

Attached Files

File	Folder	Description
CTimeFeatures.mqh	MQL5\Include\Features	Single include file: CRingBuffer and CTimeFeatures with Initialize, Update, Calculate, FeatureCount, FeatureNames
TimeFeaturesVerify.mq5	MQL5\Scripts	Script: prints tab-delimited feature values to the Experts log for comparison against Python output
TimeFeaturesDemoEA.mq5	MQL5\Experts	Minimal EA demonstrating the OnInit warmup pattern and OnTick bar-close integration