Features of the mql5 language, subtleties and tricks - page 273

New comment
 
amrali #:

The fastest replacement for the built-in TimeToStruct() function,

I recommend using this function:

About 4-5 times faster (for the version without cache).

Added it here.

1 - 37.91 ns, контрольная сумма = 231124029785  // TimeToStruct
2 - 21.67 ns, контрольная сумма = 231124029785  // amrali
2 - 15.99 ns, контрольная сумма = 231124029785  // TimeToStruct2100
2 - 10.81 ns, контрольная сумма = 231124029785  // TimeToStruct2100 fxsaber
2 - 9.38 ns, контрольная сумма = 231124029785  // TimeToStruct2100 amrali2

Indeed, your version is the fastest. Congratulations!

 

High-performance (vs built-in) time functions collected from various pages on this thread (not to get lost):

//+------------------------------------------------------------------+
//|                                                    TimeUtils.mqh |
//|                                        Copyright © 2014, Amr Ali |
//|                             https://www.mql5.com/en/users/amrali |
//+------------------------------------------------------------------+
#property version "1.00"

//+==================================================================+
//| Extract Components of datetime: Sunday, yyyy.mm.dd hh:mm         |
//| Date and Time Functions                                          |
//+==================================================================+
int   TimeSecond( const datetime t );
int   TimeMinute( const datetime t );
int   TimeHour( const datetime t );
int   TimeDay( const datetime t );
int   TimeMonth( const datetime t );
int   TimeYear( const datetime t );
int   TimeDayOfYear( const datetime t );
int   TimeDayOfWeek( const datetime t );
bool  TimeToStructFast( datetime time, MqlDateTime& dt_struct );

//+==================================================================+
//| Create datetime From Components                                  |
//+==================================================================+
datetime CreateDateTime(MqlDateTime&  dt_struct);
datetime CreateDateTime(
   const int year,           // Year
   const int mon,            // Month
   const int day,            // Day
   const int hour = 0,       // Hour
   const int min = 0,        // Minutes
   const int sec = 0         // Seconds
   );

I am planning to publish it on the codebase, maybe after adding some other useful functions.

Files:
TimeUtils.mqh  9 kb
 
fxsaber #:

Added it here.

Indeed, your version is the fastest. Congratulations!

TimeToStructFast

It the fastest I could get.

Another very fast option is using the computational calendar (fictitious calendar that starts on 1 March) and avoids the problem of dealing with leap years.

It is a new algorithm and very fast. See it here https://www.youtube.com/watch?v=0s9F4QWAl-E&t=1790s

bool TimeToCalendar(datetime time, MqlDateTime& dt_struct)
  {
   uint  K = 536895458;
   uint  L = 1468000  ;
   uint  m = 146097   ;
   uint  q = 2939745  ;
   uint  r = 2141     ;
   uint  p = 197913   ;
   ulong t = (ulong)time;
   int   n = (int)(t / 86400)                   ;  // Unix day
   uint  N = ((uint)n) + K                      ;  // Computational calendar day
   uint  a = 4 * N + 3                          ;
   uint  c = a / m                              ;
   uint  e = a % m / 4                          ;
   uint  b = 4 * e + 3                          ;
   ulong f = ((ulong)q) * b                     ;
   uint  z = (uint)(f >> 32)                    ;
   uint  h = ((uint)f) / q / 4                  ;
   uint  d = r * h + p                          ;
   uint  Y = 100 * c + z                        ;
   uint  M = d >> 16                            ;
   uint  D = ((ushort)d) / r                    ;
   uint  J = h >= 306                           ;  // Map from Computational to Gregorian calendar
   int Y_G = int((Y - L) + J)                   ;
   int M_G = int(J ? M - 12 : M)                ;
   int D_G = int(D + 1)                         ;
   int HH  = (int)((t / 3600) % 24)             ;
   int MM  = (int)((t / 60) % 60)               ;
   int SS  = (int)(t % 60)                      ;
   int dow = (n + 4) % 7                        ;
   int doy = n - ((Y_G * 5844 - 11512676) >> 4) ;
 //int doy = (((n << 2) + 2) % 1461) / 4;  // slower here!

   dt_struct.year           = Y_G;
   dt_struct.mon            = M_G;
   dt_struct.day            = D_G;
   dt_struct.hour           = HH;
   dt_struct.min            = MM;
   dt_struct.sec            = SS;
   dt_struct.day_of_week    = dow;
   dt_struct.day_of_year    = doy;

   return (true);
  }

Source: https://searchfox.org/mozilla-central/source/js/src/jsdate.cpp#263

https://github.com/torvalds/linux/blob/276010551664f73b6f1616dde471d6f0d63a73ba/kernel/time/timeconv.c#L77

 
amrali #:

High-performance (compared to inbuilt) time functions, collected from different pages of this thread (so they don't get lost):

I plan to publish it to the codebase, perhaps after adding some other useful features.

Please with a link to this thread.
 
amrali #:

This is the fastest option I could find.

Another very fast option is to use a computational calendar (a dummy calendar starting on 1 March), which avoids the leap year problem.

This is a new algorithm and very fast. You can see it here https://www.youtube.com/watch?v=0s9F4QWAl-E&t=1790s.

Source: https: //searchfox.org/mozilla-central/source/js/src/jsdate.cpp#263

https://github.com/torvalds/linux/blob/276010551664f73b6f1616dde471d6f0d63a73ba/kernel/time/timeconv.c#L77

1 - 37.72 ns, контрольная сумма = 231124376201  // TimeToStruct
2 - 21.38 ns, контрольная сумма = 231124376201  // amrali
2 - 15.79 ns, контрольная сумма = 231124376201  // TimeToStruct2100
2 - 10.71 ns, контрольная сумма = 231124376201  // TimeToStruct2100 fxsaber
2 - 9.29 ns, контрольная сумма = 231124376201  // TimeToStruct2100 amrali2
2 - 11.04 ns, контрольная сумма = 231124376201  // TimeToCalendar
 

Real live test :

TimeToStruct            2024.11.26 17:03:55.501    Core 02    EURUSD,H1: 8667407 ticks, 37169 bars generated. Environment synchronized in 0:00:00.092. Test passed in 0:01:43.931.
TimeToStruct2100Old     2024.11.26 17:10:52.902    Core 02    EURUSD,H1: 8667407 ticks, 37169 bars generated. Environment synchronized in 0:00:00.081. Test passed in 0:01:41.173.
TimeToCalendar          2024.11.26 17:25:37.672    Core 02    EURUSD,H1: 8667407 ticks, 37169 bars generated. Environment synchronized in 0:00:00.088. Test passed in 0:01:40.153.
TimeToStructFast        2024.11.26 17:31:59.339    Core 02    EURUSD,H1: 8667407 ticks, 37169 bars generated. Environment synchronized in 0:00:00.091. Test passed in 0:01:41.471.
TimeToStruct2100Last    2024.11.26 17:37:43.948    Core 02    EURUSD,H1: 8667407 ticks, 37169 bars generated. Environment synchronized in 0:00:00.084. Test passed in 0:01:40.996.
TimeToStruct2100Cache   2024.11.26 17:42:40.774    Core 02    EURUSD,H1: 8667407 ticks, 37169 bars generated. Environment synchronized in 0:00:00.091. Test passed in 0:01:40.585.

It's hot start test with all data in memory.

Total number of call of the function :

2024.11.26 17:48:38.190    Core 02    2024.09.29 23:59:59   TTS Call = 191969676.

 

Forum on trading, automated trading systems and testing trading strategies

Features of mql5 language, subtleties and techniques of work

amrali, 2024.11.26 15:58

The fastest replacement for the built-in TimeToStruct() function,

I recommend using this function:

bool TimeToStructFast(datetime time, MqlDateTime& dt_struct)
  {
//   static const int Months[13] = {0, -1, 30, 58, 89, 119, 150, 180, 211,242, 272, 303, 333};  
   static const uint Months[] = {0, 11512676, 11512180, 11511728, 11511232, 11510750, 11510256,
                                    11509774, 11509280, 11508781, 11508302, 11507806, 11507326};
   uint t       = (uint)time;
   int  n       = (int)(t / (24 * 3600));
   int  tn      = (n << 2) + 2;
   int  dow     = (n + 4) % 7;
   int  doy     = (tn % 1461) / 4;
   int  year    = (tn / 1461) + 1970; // to 2100
   int  isleap  = ((year & 3) == 0);
   int  leapadj = ((doy < (isleap + 59)) ? 0 : (2 - isleap));
   int  mon     = ((((doy + leapadj) * 12) + 373) / 367);
//   int  day     = doy - Months[mon] - (isleap && doy > 59);
   int  day     = n - (int)((year * 5844 - Months[mon]) >> 4); // https://www.mql5.com/ru/forum/170952/page254#comment_53104383
   int  HH      = (int)(t / 3600) % 24;
   int  MM      = (int)(t / 60) % 60;
   int  SS      = (int)(t % 60);

   dt_struct.year           = year;
   dt_struct.mon            = mon;
   dt_struct.day            = day;
   dt_struct.hour           = HH;
   dt_struct.min            = MM;
   dt_struct.sec            = SS;
   dt_struct.day_of_week    = dow;
   dt_struct.day_of_year    = doy;
   return (true);
  }

It's noticeably (AVX) faster that way.

 
Alain Verleyen #:

Real live test :

TimeToStruct            2024.11.26 17:03:55.501    Core 02    EURUSD,H1: 8667407 ticks, 37169 bars generated. Environment synchronized in 0:00:00.092. Test passed in 0:01:43.931.
TimeToStruct2100Old     2024.11.26 17:10:52.902    Core 02    EURUSD,H1: 8667407 ticks, 37169 bars generated. Environment synchronized in 0:00:00.081. Test passed in 0:01:41.173.
TimeToCalendar          2024.11.26 17:25:37.672    Core 02    EURUSD,H1: 8667407 ticks, 37169 bars generated. Environment synchronized in 0:00:00.088. Test passed in 0:01:40.153.
TimeToStructFast        2024.11.26 17:31:59.339    Core 02    EURUSD,H1: 8667407 ticks, 37169 bars generated. Environment synchronized in 0:00:00.091. Test passed in 0:01:41.471.
TimeToStruct2100Last    2024.11.26 17:37:43.948    Core 02    EURUSD,H1: 8667407 ticks, 37169 bars generated. Environment synchronized in 0:00:00.084. Test passed in 0:01:40.996.
TimeToStruct2100Cache   2024.11.26 17:42:40.774    Core 02    EURUSD,H1: 8667407 ticks, 37169 bars generated. Environment synchronized in 0:00:00.091. Test passed in 0:01:40.585.

It's hot start test with all data in memory.

Total number of call of the function :

2024.11.26 17:48:38.190    Core 02    2024.09.29 23:59:59   TTS Call = 191969676.

Thanks Alain for your test. The replacement for built-in TimeToStruct() is only 4-5 times faster as I reported before.

  4.29 ns, checksum = 42266650915760   // TimeToStructFast
  5.16 ns, checksum = 42266650915760   // TimeToStructFast (Cached)
 19.16 ns, checksum = 42266650915760  /// MQL's TimeToStruct()

However, the replacements for TimeYear(), TimeDayOfWeek() is at least 10-20 times faster than using MQL's TimeToStruct() 

  1.47 ns, checksum = 20510890332002139   // TimeDayOfWeek()
 18.91 ns, checksum = 20510890332002139  /// MQL's TimeToStruct()

  0.73 ns, checksum = 20512672180581744   // TimeYear()
 18.45 ns, checksum = 20512672180581744  /// MQL's TimeToStruct()

This code executes slow while using wrappers of built-in StructToTime.

Using the optimized functions really helps here.

   datetime array[];
   const int aa = CopyTime(_Symbol, PERIOD_H1, 0, Bars(_Symbol, PERIOD_H1), array);   
   if(aa<=0)
     {
      Print("Can't read quotes, error: ", _LastError);
      return;
     }
   const int n = ArraySize(array);
   ENUM_DAY_OF_WEEK weekday = -1;
   for(int i = 0; i < n; ++i)
     {
      if(TimeDayOfWeek(array[i]) > MONDAY)
         continue;
      
      int hour = TimeHour(array[i]);
      //...
      //...
     }

Another thing: decompressing (TimeToStruct) is always faster than compressing (StructToTime). Built-in StructToTime is really slow than TimeToStruct().

If your code is creating a lot of 'datetime' variables from time components, you will find CreateDateTime() is much faster than MQL's StructToTime().

I faced the problem of slow StructToTime() while constructing datetime variables for Daylight switch times in TimeZoneInfo library. Replacing it for this custom CreateDateTime() made a nice improvement in speeds.

  1.45 ns, checksum = 40671201835781217   // CreateDateTime
 34.83 ns, checksum = 40671201835781217  /// MQL's StructToTime()
 
fxsaber #:

It's noticeably (AVX) faster that way.

Unfortunately not faster than current implementation:

  X64 Regular:
  4.90 ns, checksum = 42266650915760   // TimeToStruct2100
  4.29 ns, checksum = 42266650915760   // TimeToStructFast
  5.12 ns, checksum = 42266640915760   // TimeToStructFast2
  4.28 ns, checksum = 42266650915760   // TimeToCalendar
  5.73 ns, checksum = 42266650915760   // * TimeToStruct2100 (Cached)
  5.16 ns, checksum = 42266650915760   // * TimeToStructFast (Cached)
  5.58 ns, checksum = 42266640915760   // * TimeToStructFast2 (Cached)
  5.17 ns, checksum = 42266650915760   // * TimeToCalendar (Cached)
 19.16 ns, checksum = 42266650915760  /// MQL's TimeToStruct()

  AVX:
  5.01 ns, checksum = 50711796193332   // TimeToStruct2100
  4.67 ns, checksum = 50711796193332   // TimeToStructFast
  5.43 ns, checksum = 50711786193332   // TimeToStructFast2
  4.55 ns, checksum = 50711796193332   // TimeToCalendar
  5.63 ns, checksum = 50711796193332   // * TimeToStruct2100 (Cached)
  5.40 ns, checksum = 50711796193332   // * TimeToStructFast (Cached)
  4.73 ns, checksum = 50711786193332   // * TimeToStructFast2 (Cached)
  5.57 ns, checksum = 50711796193332   // * TimeToCalendar (Cached)
 19.38 ns, checksum = 50711796193332  /// MQL's TimeToStruct()
MetaTrader 5 IC Markets (SC) x64 build 4620 started for Raw Trading Ltd
Windows 11 build 22631, 24 x 13th Gen Intel Core i7-13700KF, AVX2, 24 / 31 Gb memory, 120 / 199 Gb disk, UAC, GMT+3

 
amrali #:

Unfortunately, no faster than the current implementation:

I have a different picture. Share the source of the script, please.

1...266267268269270271272273274275276277278279280...326
New comment