ForumSections

Libraries: Calendar - page 5

New comment
BillionerClub #:

Not often, but when merging arrays, the final array comes with errors in the form of lack of sorting, because the most recent elements in the array should have a longer period of time.

I have not noticed something like this. Try to create a script to reproduce it.

Forum on trading, automated trading systems and testing trading strategies

Libraries: Calendar

BillionerClub, 2022.11.04 07:43 pm.

   int totall = Calendar+=Calendar3;

for(int i=0;i<totall;i++)
  {
   Print(TimeToString(Calendar[i].time));
  }
Add BEFORE the merge. 
Calendar.Save("Calendar.bin");
Calendar3.Save("Calendar3.bin");

And attach the generated bin-files. Then I'll be able to replicate it.

fxsaber #:
Add BEFORE the merge.

And attach the generated bin files. Then I can do it again.

Link to attached file

You will not be able to reproduce without adding to the EVENT class

  ENUM_CALENDAR_EVENT_SECTOR Sector;
  ENUM_CALENDAR_EVENT_TYPE EventType;
  ENUM_CALENDAR_EVENT_SECTOR Sector;
  ENUM_CALENDAR_EVENT_UNIT Unit;
  ENUM_CALENDAR_EVENT_MULTIPLIER Multiplier;
  ENUM_CALENDAR_EVENT_TIMEMODE TimeMode;
  ENUM_CALENDAR_EVENT_TYPE EventType;
  STRING4 CurrencySymbol;

  EVENT( void ) : time(0), Importance(CALENDAR_IMPORTANCE_NONE), id(0), EventID(0),
                  Actual(LONG_MIN), Previous(LONG_MIN), Revised(LONG_MIN), Forecast(LONG_MIN),
                  Unit(CALENDAR_UNIT_NONE), Multiplier(CALENDAR_MULTIPLIER_NONE), TimeMode(CALENDAR_TIMEMODE_DATETIME ),Sector(CALENDAR_SECTOR_NONE),EventType(CALENDAR_TYPE_HOLIDAY)
  {
Calendar.Set(NULL,CALENDAR_IMPORTANCE_MODERATE,currentday,MaxTime);
Calendar3.SetHolydays(NULL,CALENDAR_TYPE_HOLIDAY,currentday,MaxTime);
  int SetHolydays( const string sCurrency = NULL, const ENUM_CALENDAR_EVENT_TYPE EventType = CALENDAR_TYPE_HOLIDAY,
           const datetime From = -1, const datetime To = -WEEK )
  {
    int Amount = 0;

    MqlCalendarValue Values[];

    if (::CalendarValueHistory(Values, (From >= 0) ? From : ::TimeTradeServer(), (To < 0) ? ::TimeTradeServer() - To : To, NULL, sCurrency))
    {
      const int Size = ::ArrayResize(this.Events, ::ArraySize(Values));

      for (int i = 0; i < Size; i++)
      {
        if (this.Events[Amount].Set(Values[i]) && (this.Events[Amount].EventType == EventType))
        {
          Amount++;
        }
      }
    }

    return(::ArrayResize(this.Events, Amount));
  }
  
BillionerClub #:

Link to attached file 

#include <fxsaber\Calendar\Calendar.mqh>

void OnStart()
{
  CALENDAR Calendar;
  
  Calendar.Load("Calendar.bin");

  Print(Calendar.ToString());
}


2022.11.04 17:00 USD 2 Baker Hughes US Oil Rig Count(baker-hughes-us-oil-rig-count), United States(US) |  |  | 610 | 
2022.11.04 17:00 USD 2 Baker Hughes US Total Rig Count(baker-hughes-us-total-rig-count), United States(US) |  |  | 768 | 
2022.11.08 AllDay EUR 2 Economic and Financial Affairs Council Meeting(economic-financial-affairs-council-meeting), European Union(EU) |  |  |  | 
2022.11.07 ≈02:00 CNY 2 Trade Balance USD(trade-balance-usd), China(CN) |  | $83.12 B | $84.74 B | 
2022.11.07 ≈02:00 CNY 2 Trade Balance(trade-balance), China(CN) |  | ¥565.34 B | ¥573.57 B |

The reason is this. Merge sorting will only work if both calendars are sorted. Here the original calendar is not sorted by time.

fxsaber #:

the original calendar is not sorted by time.

The reason is this.

#include <fxsaber\Calendar\Calendar.mqh>

void OnStart()
{
  CALENDAR Calendar;
  
  const int Size = Calendar.Set(NULL, CALENDAR_IMPORTANCE_NONE, 0, INT_MAX); // Took the WHOLE calendar.

  for (int i = 1; i < Size; i++)
    if (Calendar[i].time < Calendar[i - 1].time)
    {
      Print("----------");
      Print(Calendar[i - 1].ToString());
      Print(Calendar[i].ToString());
    }  
}


...............

2022.11.01 AllDay EUR 0 День всех святых(all-saints-day), Италия(IT) |  |  |  | 
2022.10.31 04:00 SGD 1 Объём банковского кредитования(bank-loans), Сингапур(SG) | R$838.81 B | R$846.787 B | R$842.813 B | 
----------
2022.11.02 AllDay BRL 0 День поминовения усопших(all-souls-day), Бразилия(BR) |  |  |  | 
2022.11.01 05:30 AUD 2 Заявление Резервного Банка Австралии по процентной ставке(rba-rate-statement), Австралия(AU) |  |  |  | 
----------
2022.11.03 AllDay JPY 0 День культуры(culture-day), Япония(JP) |  |  |  | 
2022.11.02 09:00 EUR 1 Объем экспорта м/м(exports-mm), Германия(DE) | -0.5% | -0.7% | 1.6% | 2.9%
----------
2022.11.07 AllDay EUR 2 Заседание Еврогруппы(eurogroup-meeting), Европейский Союз(EU) |  |  |  | 
2022.11.04 07:00 SGD 2 Розничные продажи м/м(retail-sales-mm), Сингапур(SG) | 3.3% | 0.3% | -1.3% | -1.2%
----------
2022.11.08 AllDay EUR 2 Заседание совета по экономическим и финансовым вопросам(economic-financial-affairs-council-meeting), Европейский Союз(EU) |  |  |  | 
2022.11.07 ≈04:00 CNY 2 Торговый баланс USD(trade-balance-usd), Китай(CN) |  | $83.12 B | $84.74 B | 
----------
2022.11.11 AllDay EUR 2 Заседание совета по экономическим и финансовым вопросам(economic-financial-affairs-council-meeting), Европейский Союз(EU) |  |  |  | 
2022.11.10 09:00 NOK 2 Индекс потребительских цен м/м(cpi-mm), Норвегия(NO) |  | 0.5% | 1.4% | 

............
ZY Bug report. @BillionerClub, thanks for the tip!
fxsaber #:

the original calendar is not sorted by time.

Option to fix it.

  int Set( const string sCurrency = NULL, const ENUM_CALENDAR_EVENT_IMPORTANCE MinImportance = CALENDAR_IMPORTANCE_HIGH,
           const datetime From = -1, const datetime To = -WEEK )
  {
    int Amount = 0;

    MqlCalendarValue Values[];
    
    if (::CalendarValueHistory(Values, (From >= 0) ? From : ::TimeTradeServer(), (To < 0) ? ::TimeTradeServer() - To : To, NULL, sCurrency))
    {      
      // https://www.mql5.com/ru/forum/434546/page11#comment_43037160
      if (::TerminalInfoInteger(TERMINAL_BUILD) < 3481)) // Spell out the build where they will fix it.
        ArraySortStruct2(Values, time); // https://www.mql5.com/ru/forum/170952/page222#comment_40289584
      
      const int Size = ::ArrayResize(this.Events, ::ArraySize(Values));

      for (int i = 0; i < Size; i++)
        if (this.Events[Amount].Set(Values[i]) && (this.Events[Amount].Importance >= MinImportance))
          Amount++;
    }

    return(::ArrayResize(this.Events, Amount));
  }


In global scoping, write it.

ArraySortStruct2_Define(time) // https://www.mql5.com/ru/forum/170952/page222#comment_40289584
fxsaber #:


The reason is this. Merge sorting will work only if both calendars are sorted. Here the source calendar is not sorted by time.

Thanks. Without you I would never have figured it out )) everything looks too much correct and passed all the checks.

Please help to understand whether the calendars are correct or not.

I took known payrolls and compared several calendars. They match, it seems, by the time of news release.

// Outputs Nonfarm Payrolls data.

#include <fxsaber\Calendar\Calendar.mqh> // https://www.mql5.com/ru/code/32430

void OnStart()
{
  CALENDAR Calendar;
  
  Calendar.Set(840030016); // Nonfarm Payrolls (nonfarm-payrolls), United States (US)
  Calendar.FilterByTime(D'2019.01.01', D'2023.01.01');
  
  for (int i = Calendar.GetAmount() - 1; i >= 0; i--)
    Print(Calendar[i].ToString());
}


2022.12.02 15:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 263 K | -30 K | 261 K | 284 K
2022.11.04 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 261 K | -97 K | 263 K | 315 K
2022.10.07 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 263 K | 33 K | 315 K | 
2022.09.02 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 315 K | 156 K | 528 K | 526 K
2022.08.05 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 528 K | -19 K | 372 K | 398 K
2022.07.08 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 372 K | -229 K | 390 K | 384 K
2022.06.03 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 390 K | -19 K | 428 K | 436 K
2022.05.06 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 428 K | 317 K | 431 K | 428 K
2022.04.01 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 431 K | 80 K | 678 K | 750 K
2022.03.04 15:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 678 K | -413 K | 467 K | 481 K
2022.02.04 15:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 467 K | -192 K | 199 K | 510 K
2022.01.07 15:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 199 K | 379 K | 210 K | 249 K
2021.12.03 15:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 210 K | 330 K | 531 K | 546 K
2021.11.05 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 531 K | -54 K | 194 K | 312 K
2021.10.08 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 194 K | -302 K | 235 K | 366 K
2021.09.03 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 235 K | 23 K | 943 K | 1053 K
2021.08.06 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 943 K | 381 K | 850 K | 938 K
2021.07.02 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 850 K | 3 K | 559 K | 583 K
2021.06.04 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 559 K | -396 K | 266 K | 278 K
2021.05.07 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 266 K | -1 K | 916 K | 770 K
2021.04.02 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 916 K | 409 K | 379 K | 468 K
2021.03.05 15:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 379 K | -137 K | 49 K | 166 K
2021.02.05 15:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 49 K | -497 K | -140 K | -227 K
2021.01.08 15:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | -140 K | 207 K | 245 K | 336 K
2020.12.04 15:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 245 K | 528 K | 638 K | 610 K
2020.11.06 15:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 638 K | -76324 K | 661 K | 672 K
2020.10.02 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 661 K | 78356 K | 1371 K | 1489 K
2020.09.04 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 1371 K | -74433 K | 1763 K | 1734 K
2020.08.07 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 1763 K | 4511 K | 4800 K | 4791 K
2020.07.02 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 4800 K | -12034 K | 2509 K | 2699 K
2020.06.05 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 2509 K | -10000 K | -20500 K | -20687 K
2020.05.08 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | -20500 K | 139 K | -701 K | -870 K
2020.04.03 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | -701 K | 163 K | 273 K | 275 K
2020.03.06 15:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 273 K | 161 K | 225 K | 273 K
2020.02.07 15:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 225 K | 161 K | 145 K | 147 K
2020.01.10 15:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 145 K | 168 K | 266 K | 256 K
2019.12.06 15:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 266 K | 168 K | 128 K | 156 K
2019.11.01 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 128 K | 172 K | 136 K | 180 K
2019.10.04 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 136 K | 174 K | 130 K | 168 K
2019.09.06 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 130 K | 177 K | 164 K | 159 K
2019.08.02 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 164 K | 178 K | 224 K | 193 K
2019.07.05 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 224 K | 176 K | 75 K | 72 K
2019.06.07 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 75 K | 170 K | 263 K | 224 K
2019.05.03 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 263 K | 167 K | 196 K | 189 K
2019.04.05 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 196 K | 176 K | 20 K | 33 K
2019.03.08 15:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 20 K | 173 K | 304 K | 311 K
2019.02.01 15:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 304 K | 168 K | 312 K | 222 K
2019.01.04 15:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 312 K | 170 K | 155 K | 176 K

The tabular calendar data does not match the price data. The script shows data in the broker's time zone, so you can compare on any trading server.

It seems that forexfactori and investing calendars show the same data as MQL5.

Who else has price data that does not coincide with the calendar dates?


ZЫ If calendars lie, then how do Market Advisors bypass news in backtests?

Изменение числа занятых в несельскохозяйственном секторе - экономический индикатор США
Изменение числа занятых в несельскохозяйственном секторе - экономический индикатор США
  • www.mql5.com
Изменение числа занятых в несельскохозяйственном секторе (Nonfarm Payrolls) — показатель, отражающий количество новых рабочих мест, созданных в отчетном месяце, во всех отраслях экономики США, кроме
fxsaber #:

ZЫ If calendars lie, then how do Market Advisors get around the news in backtests?

It looks like they do lie. Automation of finding discrepancies.

// Compares Nonfarm Payrolls dates in the calendar and in the price history. Run on USD pairs.

#include <fxsaber\Calendar\Calendar.mqh> // https://www.mql5.com/ru/code/32430

// Time bar size.
double GetBarSize( const datetime Time )
{
  MqlRates Rates[];
  
  return(CopyRates(_Symbol, PERIOD_M1, iBarShift(_Symbol, PERIOD_M1, Time), 1, Rates) != 1 ? 0 : Rates[0].high - Rates[0].low);
}

// Time of the largest bar by size (by time from the array).
datetime GetMaxBarSizeTime( const datetime &Times[] )
{
  double Values[];
  
  for (int i = ArrayResize(Values, ArraySize(Times)) - 1; i >= 0; i--)
    Values[i] = GetBarSize(Times[i]);
  
  return(Times[ArrayMaximum(Values)]);
}

// Most likely time of news on price data (strong reaction).
datetime ChartNewsTime( const datetime &Time )
{
  datetime Times[3];
      
  Times[0] = Time;
  Times[1] = Times[0] - 3600;
  Times[2] = Times[0] + 3600;
  
  return(GetMaxBarSizeTime(Times));
}

void OnStart()
{
  CALENDAR Calendar;
  
  Calendar.Set(840030016); // Nonfarm Payrolls (nonfarm-payrolls), United States (US)
  Calendar.FilterByTime(D'2019.01.01', D'2023.01.01');
  
  for (int i = Calendar.GetAmount() - 1; i >= 0; i--)
  {
    const EVENT Event = Calendar[i];    
    const datetime ChartNews = ChartNewsTime(Event.time);
    
    if (Event.time != ChartNews)    
      Print(TimeToString(ChartNews) + " != " + Event.ToString());
  }
}


The discrepancies in each row below.

2022.10.07 15:30 != 2022.10.07 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 263 K | 33 K | 315 K | 
2022.09.02 15:30 != 2022.09.02 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 315 K | 156 K | 528 K | 526 K
2022.08.05 15:30 != 2022.08.05 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 528 K | -19 K | 372 K | 398 K
2022.07.08 15:30 != 2022.07.08 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 372 K | -229 K | 390 K | 384 K
2022.06.03 15:30 != 2022.06.03 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 390 K | -19 K | 428 K | 436 K
2022.05.06 15:30 != 2022.05.06 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 428 K | 317 K | 431 K | 428 K
2022.04.01 15:30 != 2022.04.01 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 431 K | 80 K | 678 K | 750 K
2021.10.08 15:30 != 2021.10.08 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 194 K | -302 K | 235 K | 366 K
2021.09.03 15:30 != 2021.09.03 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 235 K | 23 K | 943 K | 1053 K
2021.08.06 15:30 != 2021.08.06 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 943 K | 381 K | 850 K | 938 K
2021.07.02 15:30 != 2021.07.02 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 850 K | 3 K | 559 K | 583 K
2021.06.04 15:30 != 2021.06.04 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 559 K | -396 K | 266 K | 278 K
2021.05.07 15:30 != 2021.05.07 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 266 K | -1 K | 916 K | 770 K
2021.04.02 15:30 != 2021.04.02 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 916 K | 409 K | 379 K | 468 K
2020.10.02 15:30 != 2020.10.02 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 661 K | 78356 K | 1371 K | 1489 K
2020.09.04 15:30 != 2020.09.04 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 1371 K | -74433 K | 1763 K | 1734 K
2020.08.07 15:30 != 2020.08.07 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 1763 K | 4511 K | 4800 K | 4791 K
2020.07.02 15:30 != 2020.07.02 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 4800 K | -12034 K | 2509 K | 2699 K
2020.06.05 15:30 != 2020.06.05 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 2509 K | -10000 K | -20500 K | -20687 K
2020.05.08 15:30 != 2020.05.08 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | -20500 K | 139 K | -701 K | -870 K
2020.04.03 15:30 != 2020.04.03 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | -701 K | 163 K | 273 K | 275 K
2019.10.04 15:30 != 2019.10.04 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 136 K | 174 K | 130 K | 168 K
2019.09.06 15:30 != 2019.09.06 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 130 K | 177 K | 164 K | 159 K
2019.08.02 15:30 != 2019.08.02 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 164 K | 178 K | 224 K | 193 K
2019.07.05 15:30 != 2019.07.05 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 224 K | 176 K | 75 K | 72 K
2019.06.07 15:30 != 2019.06.07 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 75 K | 170 K | 263 K | 224 K
2019.05.03 15:30 != 2019.05.03 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 263 K | 167 K | 196 K | 189 K
2019.04.05 15:30 != 2019.04.05 14:30 USD 3 Nonfarm Payrolls (nonfarm-payrolls), United States (US) | 196 K | 176 K | 20 K | 33 K

On the left is the time of the news on price reaction, on the right is the calendar.


Looks like we should do an automatic corrector in the calendar dates.

123456789101112...15
New comment