work around idea of using BarsCalculated and CopyBuffer function in different timeframe

............ ............ 2019.06.17 07:17

I did search MQL5 forum but seems like iCustom is executed at incorrect sequence in OnCalculate when using it in different timeframe as the chart's base timeframe. below table in the left is when using same timeframe M5 for the chart and iCustom. right table is when using M1 timeframe in chart and using M5 timeframe in iCustom. Output behaviour of BarsCalculated and CopyBuffer function are quite different when iCustom is used in different timeframe. the difference can be found in the cells highlighted in red.

seems like this is current feature (bug) in these functions so wanted to learn ideals or what others have done to align the output to correctly.

issues

instead of getting -1 at initial load, to obtain full calculated amount.
calculated indicator values only return after a tick of new bar

  //========================================
  // property declaration
  //======================================== 
  #property indicator_chart_window
  #property indicator_buffers 1
  #property indicator_plots   1
  #property indicator_width1  1
  #property indicator_type1   DRAW_SECTION
  #property indicator_style1  STYLE_SOLID
  #property indicator_color1  Red
  #property indicator_label1  "iCustom"

  //========================================
  // enumeration declaration
  //========================================
  enum ENUM_DATA_SOURCE{
        DATA_SOURCE_M1    = 1,    // M1
        DATA_SOURCE_M5    = 5,    // M5
        DATA_SOURCE_M15   = 15,   // M15
        DATA_SOURCE_M30   = 30,   // M30
        DATA_SOURCE_H1    = 60,   // H1
        DATA_SOURCE_H4    = 240,  // H4
        DATA_SOURCE_D1    = 1440, // D1
  };

  //========================================
  // global variables declaration
  //========================================
  ENUM_TIMEFRAMES g_ZZ_eTimeframe;
  int g_ZZ_iHandler     = 0;
  int g_MTF_iBarCur     = 0;
  int g_MTF_iBarPre     = 0;
  int g_iCntOnCalculate = 0; 
  string g_BTF_sBarStat, g_MTF_sBarStat;

  //========================================
  // input parameters declaration
  //======================================== 
  // zigzag indicator parameters
  input int              i_ZZ_iDepth      = 12;                 // Depth
  input int              i_ZZ_iDeviation  = 5;                  // Deviation
  input int              i_ZZ_iBackstep   = 3;                  // Backstep
  // iCustom function parameters for MTF zigzag indicator
  input ENUM_DATA_SOURCE i_ZZ_eTimeframe  = DATA_SOURCE_M5;     
  input string           i_ZZ_sPath       = "Examples\\ZigZag"; 

  //========================================
  // Function: print to CSV (print2file)
  //========================================
  int    g_iFileHandle;
  int    g_iFileLineNo = 0; 
  int    g_iFilePageNo = 0; 
  string g_sFileName   = "Print2File_"+indicator_label1+"_BTF("+IntegerToString(Period())+")_MTF("+IntegerToString(i_ZZ_eTimeframe)+").";
  
  //================================================================================
  // Initialization function
  //================================================================================
  int OnInit(){
    Print("■ OnInit");
        g_ZZ_eTimeframe = f_eGetTimeframe(i_ZZ_eTimeframe);
    g_ZZ_iHandler = iCustom(_Symbol,g_ZZ_eTimeframe,i_ZZ_sPath,i_ZZ_iDepth,i_ZZ_iDeviation,i_ZZ_iBackstep); // load indicator to handler

    Print("BTF="+IntegerToString(_Period));
    Print("MTF="+IntegerToString(g_ZZ_eTimeframe));
    int iTest = BarsCalculated(g_ZZ_iHandler);

    //========================================
    // Function: print to CSV (print2file)
    //========================================
    g_iFileHandle = FileOpen(g_sFileName+IntegerToString(g_iFilePageNo)+".csv", FILE_CSV|FILE_WRITE); 
    
    return(INIT_SUCCEEDED);
  }

  //================================================================================
  // Deinitialization function
  //================================================================================
  void OnDeinit(const int reason){
    Print("■ OnDeinit");

    //========================================
    // Function: print to CSV (print2file)
    //========================================
    FileClose(g_iFileHandle);
  }

  //================================================================================
  // OnCalculate function
  //================================================================================ 
  int OnCalculate(
    const int       BTF_iBarCur,
    const int       BTF_iBarPre,
    const datetime &BTF_tTimeS   [],
    const double   &BTF_dOpenS   [],
    const double   &BTF_dHigS    [],
    const double   &BTF_dLowS    [],
    const double   &BTF_dCloseS  [],
    const long     &BTF_lTickVolS[],
    const long     &BTF_lRealVolS[],
    const int      &BTF_iSpreadS []){

    g_iCntOnCalculate++; // [for debugging]

    MqlRates stMTFS[];
    int MTF_iHistBarQty = iBars(_Symbol, g_ZZ_eTimeframe);
    g_MTF_iBarCur       = CopyRates(_Symbol, g_ZZ_eTimeframe, 0, MTF_iHistBarQty, stMTFS);
    
    Print("HistBarQty    ="+IntegerToString(MTF_iHistBarQty));
    Print("CopyRateBarQty="+IntegerToString(g_MTF_iBarCur));

//---

    double ZZ_dBufferS[];
    int MTF_iIndCalBarQty     = BarsCalculated(g_ZZ_iHandler);
    int MTF_iCopyBufferBarQty = CopyBuffer(g_ZZ_iHandler, 0, 0, MTF_iHistBarQty, ZZ_dBufferS);

    Print("IndCalBarQty     ="+IntegerToString(MTF_iIndCalBarQty));
    Print("CopyBufferBarQty ="+IntegerToString(MTF_iCopyBufferBarQty));
        
    if(BTF_iBarCur!=BTF_iBarPre && BTF_iBarPre==0){
      g_BTF_sBarStat = "Initial";
           
    }else if(BTF_iBarCur!=BTF_iBarPre && BTF_iBarPre!=0){
      g_BTF_sBarStat = "New Bar";

    }else if(BTF_iBarCur==BTF_iBarPre){
      g_BTF_sBarStat = "Old Bar";
    }

    if(g_MTF_iBarCur!=g_MTF_iBarPre && g_MTF_iBarPre==0){
      g_MTF_sBarStat = "Initial";
           
    }else if(g_MTF_iBarCur!=g_MTF_iBarPre && g_MTF_iBarPre!=0){
      g_MTF_sBarStat = "New Bar";

    }else if(g_MTF_iBarCur==g_MTF_iBarPre){
      g_MTF_sBarStat = "Old Bar";
    }

    
    f_vPrint2File(BTF_iBarCur, BTF_iBarPre, MTF_iHistBarQty, MTF_iIndCalBarQty, MTF_iCopyBufferBarQty, BTF_tTimeS, stMTFS);

    g_MTF_iBarPre = g_MTF_iBarCur; // return value of iMTF_BarCur for next call
    return(BTF_iBarCur);
  }

  //================================================================================
  // Custom function
  //================================================================================

  //========================================
  // Function: Get the timeframe of the data source 
  //========================================
  ENUM_TIMEFRAMES f_eGetTimeframe(ENUM_DATA_SOURCE eDataSource){
        switch (eDataSource){
                case DATA_SOURCE_M1:    return(PERIOD_M1);
                case DATA_SOURCE_M5:    return(PERIOD_M5);
                case DATA_SOURCE_M15:   return(PERIOD_M15);
                case DATA_SOURCE_M30:   return(PERIOD_M30);
                case DATA_SOURCE_H1:    return(PERIOD_H1);
                case DATA_SOURCE_H4:    return(PERIOD_H4);
                case DATA_SOURCE_D1:    return(PERIOD_D1);
                default: 
                  Print("Error: Invalid eDataSource value used in f_eGetTimeframe.");
                  return(-1);
        }
  }  

  //================================================================================
  // Function: print to CSV (print2file)
  //================================================================================
  // find a way to declare gobal variable in function so all below can contain in one location for easy maintenance. use OOP
  void f_vPrint2File(
    int BTF_iBarCur, 
    int BTF_iBarPre, 
    int MTF_iHistBarQty, 
    int MTF_iIndCalBarQty, 
    int MTF_iCopyBufferBarQty, 
    const datetime &BTF_tTimeS[],
    const MqlRates &stMTFS[]){

    // create new file if line count is more than 1,000,000 lines. preventing oversize and overload to open the file
    if(g_iFileLineNo>1000000){
      FileClose(g_iFileHandle);
      g_iFilePageNo++;
      g_iFileLineNo = 0;
      g_iFileHandle = FileOpen(g_sFileName+IntegerToString(g_iFilePageNo)+".csv",FILE_CSV|FILE_WRITE);
    }
    
    // add header only at new file
    if(g_iFileLineNo==0){
      // print header to file
      FileWrite(g_iFileHandle,
        "g_iCntOnCalculate",
        "g_BTF_sBarStat",
        "g_MTF_sBarStat",
        "", //---
        "BTF_iBarCur",
        "BTF_iBarPre",
        "g_MTF_iBarCur",
        "g_MTF_iBarPre",
        "", //---
        "MTF_iHistBarQty",
        "g_MTF_iBarCur",
        "", //---
        "MTF_iIndCalBarQty",
        "MTF_iCopyBufferBarQty",
        "", //---
        "BTF_tTimeS[BTF_iBarCur-1]",
//        "BTF_tTimeS[BTF_iBarPre-2]",
        "stMTFS[g_MTF_iBarCur-1].time"
//        "stMTFS[g_MTF_iBarPre-1].time"
      );
      g_iFileLineNo++;    
    }

    // add debug data
    FileWrite(g_iFileHandle,
        g_iCntOnCalculate,
        g_BTF_sBarStat,
        g_MTF_sBarStat,
        "", //---
        BTF_iBarCur,
        BTF_iBarPre,
        g_MTF_iBarCur,
        g_MTF_iBarPre,
        "", //---
        MTF_iHistBarQty,
        g_MTF_iBarCur,
        "", //---
        MTF_iIndCalBarQty,
        MTF_iCopyBufferBarQty,
        "", //---
        BTF_tTimeS[BTF_iBarCur-1],
//        BTF_tTimeS[BTF_iBarPre-2],
        stMTFS[g_MTF_iBarCur-1].time
//        stMTFS[g_MTF_iBarPre-1].time
    );
    g_iFileLineNo++;    
  }

Seng Joo Thio 2019.06.17 08:11 #1

............ ............:

seems like this is current feature (bug) in these functions so wanted to learn ideals or what others have done to align the output to correctly.

issues

instead of getting -1 at initial load, to obtain full calculated amount.
calculated indicator values only return after a tick of new bar

If it takes X microseconds to access 100,000 bars of data, it'll probably take between X and 2X microseconds to access 2 x 100,000 bars of data. So having to wait slightly longer when you're also accessing data from MTF should be expected.

You can make your OnCalculate() do nothing and return 0 when BarsCalculate(g_ZZ_iHandler) is less than the bar count.

Errors, bugs, questions Problems with ERR_HISTORY_WILL_UPDATED (4066 Am I wrong or

............ ............ 2019.06.17 18:46 #2

Seng Joo Thio:

You can make your OnCalculate() do nothing and return 0 when BarsCalculate(g_ZZ_iHandler) is less than the bar count.

appreciated for a fast feedback : )

i removed it from sample code but i actually did make it sleep for 50 and for loop for 10 times but it was same result so i removed it. are you saying you have tried my sample code with slep and was able to successful not getting -1 or slip sequence? if so, how long did you got it sleep?

Buffer to open trade [Archive!] FOREX - Trends, Looking for MA cross

Seng Joo Thio 2019.06.17 21:54 #3

............ ............:

appreciated for a fast feedback : )

Don't use sleep because it has no effect on indicators. And don't use loop, because it'll still be hogging the thread (and not giving the system any chance to complete calculation for the indicators you need).

You just have to return(0) instead of return(BTF_iBarCur), to indicate that no computation was done this time round, and do everything in the next tick - afterall the next tick probably comes before your're done with your long loop (or sleep, assuming that works)...

Help on Trix Trend RefreshRates, init() and start() Recursive Polling

............ ............ 2019.06.18 01:42 #4

i have already tested it works on next tick but that is not what I need.

if you check my excel table.

left side of the table show when base chart timeframe (M5) and indicator timeframe (M5) are same, therefore there are no slipping.

right side of the table show when base chart timeframe (M1) and indicator timeframe (M5) are different, therefore there are slipping on the result of the calculated indicator values. (in pink cells)

is there a way not to get -1 at initial tick load and also slipping?

slipping means indicator returns update values on next tick of new bar.

Indicator that show results EA using indicator Questions from a "dummy"

Seng Joo Thio 2019.06.18 02:13 #5

............ ............:

i have already tested it works on next tick but that is not what I need.

if you check my excel table.

left side of the table show when base chart timeframe (M5) and indicator timeframe (M5) are same, therefore there are no slipping.

right side of the table show when base chart timeframe (M1) and indicator timeframe (M5) are different, therefore there are slipping on the result of the calculated indicator values. (in pink cells)

is there a way not to get -1 at initial tick load and also slipping?

slipping means indicator returns update values on next tick of new bar.

There is no slippage - check the times, not bar counts.

Alain Verleyen 2019.06.18 03:55 #6

...

is there a way not to get -1 at initial tick load and also slipping?

No.

slipping means indicator returns update values on next tick of new bar.

Of course you have a "slipping", the ZZ indicator and your indicator are calculated in the same thread, so they are calculated in sequence one after other.

Your indicator is calculated first, ZZ after, so you always have 1 ticks shift. That's why you have BarsCalculated(), to check what will have as values. You can use it and just wait the next tick to get new bar value of ZZ. This log shows situation on a new MTF (M5) bar :

2019.06.17 23:37:23.916    315917 (EURUSD,M1)    tick#1     M5 Bars = 1000008 => 1000008   BarsCalculated = -1 => -1
2019.06.17 23:37:31.461    315917 (EURUSD,M1)    tick#2     M5 Bars = 1000008 => 1000008   BarsCalculated = 1000008 => 1000008
2019.06.17 23:40:00.372    315917 (EURUSD,M1)    tick#30   M5 Bars = 1000009 => 1000009   BarsCalculated = 1000008 => 1000008
2019.06.17 23:40:49.535    315917 (EURUSD,M1)    tick#31   M5 Bars = 1000009 => 1000009   BarsCalculated = 1000009 => 1000009
2019.06.17 23:45:00.882    315917 (EURUSD,M1)    tick#116 M5 Bars = 1000010 => 1000010   BarsCalculated = 1000009 => 1000009
2019.06.17 23:45:01.922    315917 (EURUSD,M1)    tick#117 M5 Bars = 1000010 => 1000010   BarsCalculated = 1000010 => 1000010

If this "tick" shift is not tolerable for you (I am curious to know why in this case ?), you have two choices :

Make sure ZZ is calculated before your indicator : I am not sure this is possible.
Calculate all in one indicator, so you will have total control.

P.S: I am suggesting you, you post code on the forum and request for help, to not change standard variable names like OnCalculate rates_total, prev_calculated, etc... It's very annoying.

P.S²: Copying all history rates or indicator's buffer on each tick is a bit insane.

Errors, bugs, questions Need help with two Questions from Beginners MQL5

............ ............ 2019.06.25 10:54 #7

Hi Alan

Thank you for your feedback.

No.

Of course you have a "slipping", the ZZ indicator and your indicator are calculated in the same thread, so they are calculated in sequence one after other.

If this "tick" shift is not tolerable for you (I am curious to know why in this case ?), you have two choices :

Make sure ZZ is calculated before your indicator : I am not sure this is possible.
Calculate all in one indicator, so you will have total control.

I agree your insight as I also read other comment and assumed it may be the case. It was good to confirm from expert's view.

I actually did #2 suggestion before I used iCustom but found out it will be easier by using iCustom but come across this issue and wanted to confirm my understanding was correct.

P.S: I am suggesting you, you post code on the forum and request for help, to not change standard variable names like OnCalculate rates_total, prev_calculated, etc... It's very annoying.

Noted, from next time I will make sure to keep it as standard naming. Apology and thank you for taking time to read my code.

P.S²: Copying all history rates or indicator's buffer on each tick is a bit insane.

thank you for your comment. I am not intend to call it every tick as it was for testing purpose.

Green Long Bar in Indicator does not use Open Only back testing

New comment