TimeToCalendar in AVX-compiler is faster than any other implementation - General

fxsaber 2024.11.27 16:57 #2741

fxsaber #:

Correct and faster now.

Random.

Compiler Version: 4699 X64 Regular, optimization - true
Intel Core i7-2700 K  @ 3.50 GHz, AVX
With hours (dt.hour+ dt.min+ dt.sec - on), random datetimes[].
1970.01.01 00:00:04 - 2097.11.29 23:59:24
 9.97 ns, checksum = 1287384081405080   // TimeToStruct2100
 9.65 ns, checksum = 1287384081405080   // TimeToStructFast
 9.81 ns, checksum = 1287384081405080   // TimeToStructFast_fxsaber
11.40 ns, checksum = 1287384081405080   // TimeToCalendar
21.79 ns, checksum = 1287384081405080   // TimeToJulian
39.66 ns, checksum = 1287384081405080  /// MQL's TimeToStruct()

Compiler Version: 4699 AVX, optimization - true
Intel Core i7-2700 K  @ 3.50 GHz, AVX
With hours (dt.hour+ dt.min+ dt.sec - on), random datetimes[].
1970.01.01 00:00:09 - 2097.11.29 23:58:42
10.34 ns, checksum = 1258378892513111   // TimeToStruct2100
10.23 ns, checksum = 1258378892513111   // TimeToStructFast
10.17 ns, checksum = 1258378892513111   // TimeToStructFast_fxsaber
12.07 ns, checksum = 1258378892513111   // TimeToCalendar
22.58 ns, checksum = 1258378892513111   // TimeToJulian
38.64 ns, checksum = 1258378892513111  /// MQL's TimeToStruct()

Compiler Version: 4699 X64 Regular, optimization - true
Intel Core i7-2700 K  @ 3.50 GHz, AVX
Without hours (dt.hour+ dt.min+ dt.sec - off), random datetimes[].
1970.01.01 00:00:12 - 2097.11.29 23:59:10
 7.95 ns, checksum = 1236392076157598   // TimeToStruct2100
 7.62 ns, checksum = 1236392076157598   // TimeToStructFast
 7.59 ns, checksum = 1236392076157598   // TimeToStructFast_fxsaber
 8.45 ns, checksum = 1236392076157598   // TimeToCalendar
17.71 ns, checksum = 1236392076157598   // TimeToJulian
38.96 ns, checksum = 1236392076157598  /// MQL's TimeToStruct()

Compiler Version: 4699 AVX, optimization - true
Intel Core i7-2700 K  @ 3.50 GHz, AVX
Without hours (dt.hour+ dt.min+ dt.sec - off), random datetimes[].
1970.01.01 00:00:02 - 2097.11.29 23:59:39
 8.29 ns, checksum = 1256777674228719   // TimeToStruct2100
 7.83 ns, checksum = 1256777674228719   // TimeToStructFast
 7.99 ns, checksum = 1256777674228719   // TimeToStructFast_fxsaber
 9.19 ns, checksum = 1256777674228719   // TimeToCalendar
 6.85 ns, checksum = 1256777674228719   // TimeToJulian
38.48 ns, checksum = 1256777674228719  /// MQL's TimeToStruct()

Not random.

Compiler Version: 4699 X64 Regular, optimization - true
Intel Core i7-2700 K  @ 3.50 GHz, AVX
With hours (dt.hour+ dt.min+ dt.sec - on), not random datetimes[].
1970.01.01 00:00:00 - 2099.11.29 23:59:33
 9.38 ns, checksum = 706250935681392   // TimeToStruct2100
 9.65 ns, checksum = 706250935681392   // TimeToStructFast
 8.48 ns, checksum = 706250935681392   // TimeToStructFast_fxsaber
11.45 ns, checksum = 706250935681392   // TimeToCalendar
21.59 ns, checksum = 706250935681392   // TimeToJulian
 6.38 ns, checksum = 706250935681392  /// MQL's TimeToStruct()

Compiler Version: 4699 AVX, optimization - true
Intel Core i7-2700 K  @ 3.50 GHz, AVX
With hours (dt.hour+ dt.min+ dt.sec - on), not random datetimes[].
1970.01.01 00:00:00 - 2099.11.29 23:59:33
 9.75 ns, checksum = 706250935681392   // TimeToStruct2100
10.31 ns, checksum = 706250935681392   // TimeToStructFast
 8.93 ns, checksum = 706250935681392   // TimeToStructFast_fxsaber
12.11 ns, checksum = 706250935681392   // TimeToCalendar
22.81 ns, checksum = 706250935681392   // TimeToJulian
 6.51 ns, checksum = 706250935681392  /// MQL's TimeToStruct()

Compiler Version: 4699 X64 Regular, optimization - true
Intel Core i7-2700 K  @ 3.50 GHz, AVX
Without hours (dt.hour+ dt.min+ dt.sec - off), not random datetimes[].
1970.01.01 00:00:00 - 2099.11.29 23:59:33
 7.07 ns, checksum = 706253275674880   // TimeToStruct2100
 7.55 ns, checksum = 706253275674880   // TimeToStructFast
 6.47 ns, checksum = 706253275674880   // TimeToStructFast_fxsaber
 8.46 ns, checksum = 706253275674880   // TimeToCalendar
17.92 ns, checksum = 706253275674880   // TimeToJulian
 5.93 ns, checksum = 706253275674880  /// MQL's TimeToStruct()

Compiler Version: 4699 AVX, optimization - true
Intel Core i7-2700 K  @ 3.50 GHz, AVX
Without hours (dt.hour+ dt.min+ dt.sec - off), not random datetimes[].
1970.01.01 00:00:00 - 2099.11.29 23:59:33
 7.57 ns, checksum = 706253275674880   // TimeToStruct2100
 7.86 ns, checksum = 706253275674880   // TimeToStructFast
 6.75 ns, checksum = 706253275674880   // TimeToStructFast_fxsaber
 9.16 ns, checksum = 706253275674880   // TimeToCalendar
 6.83 ns, checksum = 706253275674880   // TimeToJulian
 5.94 ns, checksum = 706253275674880  /// MQL's TimeToStruct()

Magic.

We remove this line in the source (see attachment).

   Print((string)t[ArrayMinimum(t)] + " - " + (string)t[ArrayMaximum(t)]);

Compiler Version: 4699 AVX, optimization - true
Intel Core i7-2700 K  @ 3.50 GHz, AVX
Without hours (dt.hour+ dt.min+ dt.sec - off), random datetimes[].
TimeToCalendar's magic
 7.91 ns, checksum = 1238992342013450   // TimeToStruct2100
 7.62 ns, checksum = 1238992342013450   // TimeToStructFast
 7.79 ns, checksum = 1238992342013450   // TimeToStructFast_fxsaber
 4.02 ns, checksum = 1238992342013450   // TimeToCalendar
 6.87 ns, checksum = 1238992342013450   // TimeToJulian
38.02 ns, checksum = 1238992342013450  /// MQL' s TimeToStruct()

Compiler Version: 4699 AVX, optimization - true
Intel Core i7-2700 K  @ 3.50 GHz, AVX
Without hours (dt.hour+ dt.min+ dt.sec - off), not random datetimes[].
TimeToCalendar's magic
 7.29 ns, checksum = 706253275674880   // TimeToStruct2100
 7.60 ns, checksum = 706253275674880   // TimeToStructFast
 6.59 ns, checksum = 706253275674880   // TimeToStructFast_fxsaber
 4.00 ns, checksum = 706253275674880   // TimeToCalendar
 6.80 ns, checksum = 706253275674880   // TimeToJulian
 5.69 ns, checksum = 706253275674880  /// MQL' s TimeToStruct()

TimeToCalendar in AVX-compiler is very much ahead of any other implementation.

Files:

TimeToStruct_benchmarks.mq5 30 kb

Alain Verleyen 2024.11.27 19:01 #2742

fxsaber #:

Random.

Not random.

Magic.

We remove this line in the source (see attachment).

TimeToCalendar in AVX-compiler is very much ahead of any other implementation.

AVX2 is slower ?

fxsaber 2024.11.27 19:06 #2743

Alain Verleyen #:

Is AVX2 running slower?

I don't have such a processor to test.

The source is working, you can try it.

Alain Verleyen 2024.11.27 19:14 #2744

fxsaber #:

I don't have such a processor to test.

The source is working, you can try it.

Ah ok, I will do it.

Alain Verleyen 2024.11.27 19:21 #2745

Comparison of the 3 compilation mode at my disposal (I don't have AVX512).

Compiler Version: 4702 AVX2 + FMA3, optimization - true
Intel Core i7-9750H @ 2.60GHz, AVX2 + FMA3
Without hours (dt.hour+ dt.min+ dt.sec - off), random datetimes[].
1970.01.01 00:01:44 - 2097.11.29 23:59:41
6.29 ns, checksum = 1269354861838493   // TimeToStruct2100
5.90 ns, checksum = 1269354861838493   // TimeToStructFast
6.49 ns, checksum = 1269354861838493   // TimeToStructFast_fxsaber
3.60 ns, checksum = 1269354861838493   // TimeToCalendar
6.47 ns, checksum = 1269354861838493   // TimeToJulian
31.65 ns, checksum = 1269354861838493 /// MQL's TimeToStruct()

Compiler Version: 4702 AVX, optimization - true
Intel Core i7-9750H @ 2.60GHz, AVX2 + FMA3
Without hours (dt.hour+ dt.min+ dt.sec - off), random datetimes[].
1970.01.01 00:00:47 - 2097.11.29 23:59:41
6.30 ns, checksum = 1286714313014774   // TimeToStruct2100
5.92 ns, checksum = 1286714313014774   // TimeToStructFast
6.40 ns, checksum = 1286714313014774   // TimeToStructFast_fxsaber
6.46 ns, checksum = 1286714313014774   // TimeToCalendar
6.11 ns, checksum = 1286714313014774   // TimeToJulian
31.89 ns, checksum = 1286714313014774 /// MQL's TimeToStruct()

Compiler Version: 4702 X64 Regular, optimization - true
Intel Core i7-9750H @ 2.60GHz, AVX2 + FMA3
Without hours (dt.hour+ dt.min+ dt.sec - off), random datetimes[].
1970.01.01 00:00:09 - 2097.11.29 23:58:42
6.13 ns, checksum = 1257992266603185   // TimeToStruct2100
5.69 ns, checksum = 1257992266603185   // TimeToStructFast
6.37 ns, checksum = 1257992266603185   // TimeToStructFast_fxsaber
6.27 ns, checksum = 1257992266603185   // TimeToCalendar
13.06 ns, checksum = 1257992266603185   // TimeToJulian
31.63 ns, checksum = 1257992266603185 /// MQL's TimeToStruct()

PREDICT time period ASCTrend system Result of the previous

Edgar Akhmadeev 2024.11.27 19:51 #2746

2024.11.27 21:44:37.971 TimeToStruct_benchmarks (BTCUSDT,M1)    Compiler Version: 4695 X64 Regular, optimization - true
2024.11.27 21:44:37.971 TimeToStruct_benchmarks (BTCUSDT,M1)    AMD Ryzen 3 PRO 3200 GE w/ Radeon Vega, AVX2 + FMA3
2024.11.27 21:44:37.971 TimeToStruct_benchmarks (BTCUSDT,M1)    Without hours (dt.hour+ dt.min+ dt.sec - off), random datetimes[].
2024.11.27 21:44:39.630 TimeToStruct_benchmarks (BTCUSDT,M1)    1970.01.01 00:00:03 - 2097.11.29 23:59:41
2024.11.27 21:44:40.659 TimeToStruct_benchmarks (BTCUSDT,M1)     6.78 ns, checksum = 1282374964710342   // TimeToStruct2100
2024.11.27 21:44:41.651 TimeToStruct_benchmarks (BTCUSDT,M1)     6.53 ns, checksum = 1282374964710342   // TimeToStructFast
2024.11.27 21:44:42.705 TimeToStruct_benchmarks (BTCUSDT,M1)     6.94 ns, checksum = 1282374964710342   // TimeToStructFast_fxsaber
2024.11.27 21:44:43.747 TimeToStruct_benchmarks (BTCUSDT,M1)     6.86 ns, checksum = 1282374964710342   // TimeToCalendar
2024.11.27 21:44:45.990 TimeToStruct_benchmarks (BTCUSDT,M1)    14.77 ns, checksum = 1282374964710342   // TimeToJulian
2024.11.27 21:44:51.837 TimeToStruct_benchmarks (BTCUSDT,M1)    38.51 ns, checksum = 1282374964710342  /// MQL's TimeToStruct()
2024.11.27 21:46:10.525 TimeToStruct_benchmarks (BTCUSDT,M1)    
2024.11.27 21:46:10.525 TimeToStruct_benchmarks (BTCUSDT,M1)    Compiler Version: 4695 AVX, optimization - true
2024.11.27 21:46:10.525 TimeToStruct_benchmarks (BTCUSDT,M1)    AMD Ryzen 3 PRO 3200 GE w/ Radeon Vega, AVX2 + FMA3
2024.11.27 21:46:10.525 TimeToStruct_benchmarks (BTCUSDT,M1)    Without hours (dt.hour+ dt.min+ dt.sec - off), random datetimes[].
2024.11.27 21:46:12.188 TimeToStruct_benchmarks (BTCUSDT,M1)    1970.01.01 00:00:47 - 2097.11.29 23:59:41
2024.11.27 21:46:13.287 TimeToStruct_benchmarks (BTCUSDT,M1)     7.24 ns, checksum = 1290059549437091   // TimeToStruct2100
2024.11.27 21:46:14.328 TimeToStruct_benchmarks (BTCUSDT,M1)     6.85 ns, checksum = 1290059549437091   // TimeToStructFast
2024.11.27 21:46:15.437 TimeToStruct_benchmarks (BTCUSDT,M1)     7.30 ns, checksum = 1290059549437091   // TimeToStructFast_fxsaber
2024.11.27 21:46:16.599 TimeToStruct_benchmarks (BTCUSDT,M1)     7.66 ns, checksum = 1290059549437091   // TimeToCalendar
2024.11.27 21:46:17.405 TimeToStruct_benchmarks (BTCUSDT,M1)     5.31 ns, checksum = 1290059549437091   // TimeToJulian
2024.11.27 21:46:23.284 TimeToStruct_benchmarks (BTCUSDT,M1)    38.72 ns, checksum = 1290059549437091  /// MQL's TimeToStruct()
2024.11.27 21:46:56.638 TimeToStruct_benchmarks (BTCUSDT,M1)    
2024.11.27 21:46:56.638 TimeToStruct_benchmarks (BTCUSDT,M1)    Compiler Version: 4695 AVX2 + FMA3, optimization - true
2024.11.27 21:46:56.638 TimeToStruct_benchmarks (BTCUSDT,M1)    AMD Ryzen 3 PRO 3200 GE w/ Radeon Vega, AVX2 + FMA3
2024.11.27 21:46:56.638 TimeToStruct_benchmarks (BTCUSDT,M1)    Without hours (dt.hour+ dt.min+ dt.sec - off), random datetimes[].
2024.11.27 21:46:58.296 TimeToStruct_benchmarks (BTCUSDT,M1)    1970.01.01 00:00:03 - 2097.11.29 23:59:19
2024.11.27 21:46:59.388 TimeToStruct_benchmarks (BTCUSDT,M1)     7.19 ns, checksum = 1242845810931935   // TimeToStruct2100
2024.11.27 21:47:00.434 TimeToStruct_benchmarks (BTCUSDT,M1)     6.89 ns, checksum = 1242845810931935   // TimeToStructFast
2024.11.27 21:47:01.518 TimeToStruct_benchmarks (BTCUSDT,M1)     7.14 ns, checksum = 1242845810931935   // TimeToStructFast_fxsaber
2024.11.27 21:47:02.044 TimeToStruct_benchmarks (BTCUSDT,M1)     3.47 ns, checksum = 1242845810931935   // TimeToCalendar
2024.11.27 21:47:02.899 TimeToStruct_benchmarks (BTCUSDT,M1)     5.63 ns, checksum = 1242845810931935   // TimeToJulian
2024.11.27 21:47:08.741 TimeToStruct_benchmarks (BTCUSDT,M1)    38.47 ns, checksum = 1242845810931935  /// MQL's TimeToStruct()

If interested... I apologise, I didn't clean it up.

fxsaber 2024.11.27 20:05 #2747

Alain Verleyen #:

Comparison of the 3 compilation modes available to me (I don't have AVX512).

Edgar Akhmadeev #:
If interested... I apologise, I didn't clean it up.

You can see well that there is a strong dependence on compiler settings. AVX2+FMA3 significantly affects TimeToCalendar.

Apparently, there are some acceleration laws when a lot of such simple actions is more favourable than few complex expressions.

A lot of simple ones.

Forum on trading, automated trading systems and testing trading strategies.

Peculiarities of the mql5 language, subtleties and techniques of work

amrali, 2024.11.26 18:33

   uint  K = 536895458;
   uint  L = 1468000  ;
   uint  m = 146097   ;
   uint  q = 2939745  ;
   uint  r = 2141     ;
   uint  p = 197913   ;
   ulong t = (ulong)time;
   int   n = (int)(t / 86400)                   ;  // Unix day
   uint  N = ((uint)n) + K                      ;  // Computational calendar day
   uint  a = 4 * N + 3                          ;
   uint  c = a / m                              ;
   uint  e = a % m / 4                          ;
   uint  b = 4 * e + 3                          ;
   ulong f = ((ulong)q) * b                     ;
   uint  z = (uint)(f >> 32)                    ;
   uint  h = ((uint)f) / q / 4                  ;
   uint  d = r * h + p                          ;
   uint  Y = 100 * c + z                        ;
   uint  M = d >> 16                            ;
   uint  D = ((ushort)d) / r                    ;
   uint  J = h >= 306                           ;  // Map from Computational to Gregorian calendar
   int Y_G = int((Y - L) + J)                   ;
   int M_G = int(J ? M - 12 : M)                ;
   int D_G = int(D + 1)                         ;

Few complex.

Forum on trading, automated trading systems and testing trading strategies

Peculiarities of mql5 language, subtleties and techniques of work

fxsaber, 2024.11.27 14:32

   static const int Months[] = {0, 11512692, 11512196, 11511744, 11511248, 11510766, 11510272,
                                   11509790, 11509296, 11508797, 11508318, 11507822, 11507342};
   uint t       = (uint)time;
   int  n       = (int)(t / (24 * 3600));
   int  tn      = (n << 2) + 2;
   int  doy     = (tn % 1461) / 4;
   int  year    = (tn / 1461) + 1970; // to 2100
   int  mon     = (doy < 59) ? ((doy + 1) >> 5) + 1 : (((doy - !(year & 3)) * 67 + 2209 ) >> 11);   
   int  day     = n - (int)((year * 5844 - Months[mon]) >> 4);

It is not clear how to understand from the code which of the bottom will be faster.

Errors, bugs, questions How to Start with TimeTradeServer analogue for MT4

Maxim Kuznetsov 2024.11.27 20:19 #2748

for the sake of clarity, it is necessary to compare it with the system/library functions. (there are several variants in the system, which of them is more "basic" I find it difficult to answer, those who are interested will find out).

struct struct_tm {
   int tm_sec;   // секунды
   int tm_min;   // минуты
   int tm_hour;  // часы
   int tm_mday;  // дни
   int tm_mon;   // месяц (от 0 январь)
   int tm_year;  // год (от 1900)
   int tm_wday;  // день недели
   int tm_yday;  // день года
   int tm_isdst; // флаг летнего времени   
};
#import "ucrtbase.dll"
   ulong _localtime64_s(struct_tm &res,datetime &time);
#import 
void OnStart()
{
   struct_tm tm;
   datetime time=TimeLocal();
   _localtime64_s(tm,time);
   PrintFormat("tm_year=%d tm_mon=%d tm_day=%d tm_hour=%d tm_min=%d tm_sec=%d",
      tm.tm_year+1900,tm.tm_mon+1,tm.tm_mday,tm.tm_hour,tm.tm_min,tm.tm_sec);
}

In general, there are only 2 functions for converting time into structures: (1) the standard one from the authors of the terminal and MQL, (2) from the authors of the OS and system libraries. And almost exactly the 1st one directly or indirectly causes the 2nd one.

PS/ I will never use and advise anyone not to use their homemade "fast" analogues. All speed measurements are just a sport for the sake of competition and "whose confu is cooler" :-)

Difficulties of translation :) My approach. The core MT4 iMAOnArray and iBandsOnArray

Edgar Akhmadeev 2024.11.27 20:22 #2749

fxsaber #:

It is not clear how to understand from the code which of the bottoms will be faster.

Eh, where are the times of assembly language programming? I used to calculate speed by knowing the number of clock cycles of each instruction. I had to do it when in the 80s I wrote a driver for our laboratory audio card and a speech synthesiser for GTS.

And on the matter - if you calculate the number of mat. operations and estimate the speed of each type ( * / + - - )?

Help write a linear Is <11 faster than Errors, bugs, questions

fxsaber 2024.11.27 20:24 #2750

Edgar Akhmadeev #:

And to the point - if you count the number of mat operations and estimate the speed of each type ( * / + - - )?

I did. I didn't see any reason for such a difference.

Features of the mql5 language, subtleties and tricks - page 275

Random.

Not random.

Magic.

Random.

Not random.

Magic.

A lot of simple ones.

Few complex.