iTime and iBarShift return wrong values when back test for multi timeframes

New comment
 

Hi Guys,

Can anyone please help. I wrote a multi timeframe indicator and it runs correctly when loading into a static chart. However when doing a back test it retrieved wrong values. Please review below code and results.

How does that happen? This seems to be a MT4 bug. Anyone has experience how to get the correct bar open time for any other timeframe?

Thanks


#property strict
#property indicator_chart_window

int OnInit(){   
   return(INIT_SUCCEEDED);
  }
//+------------------------------------------------------------------+
//| Custom indicator iteration function                              |
//+------------------------------------------------------------------+
int OnCalculate(const int rates_total,
                const int prev_calculated,
                const datetime &time[],
                const double &open[],
                const double &high[],
                const double &low[],
                const double &close[],
                const long &tick_volume[],
                const long &volume[],
                const int &spread[])
  {
//---
   int counted_bars = IndicatorCounted();
   if (counted_bars < 0) return (-1);
   if (counted_bars > 0) counted_bars--;
   int limit = Bars - counted_bars-1;   
   for (int bar=limit;bar>=0;bar--){    
      datetime dt=Time[bar];
      datetime Time_H1=iTime(Symbol(),PERIOD_H1,iBarShift(Symbol(),PERIOD_H1,dt));
      datetime Time_H4=iTime(Symbol(),PERIOD_H4,iBarShift(Symbol(),PERIOD_H4,dt));
      datetime Time_D1=iTime(Symbol(),PERIOD_D1,iBarShift(Symbol(),PERIOD_D1,dt));
      datetime Time_MN1=iTime(Symbol(),PERIOD_MN1,iBarShift(Symbol(),PERIOD_MN1,dt));
      Print(bar,": dt=",dt," Time_H1=",Time_H1," Time_H4=",Time_H4," Time_D1=",Time_D1," Time_MN1=",Time_MN1);
   }
   return(rates_total);
  }
//+------------------------------------------------------------------+


The results:

Results

 

It appears to be correct.

There isn't an H4 or D1 bar that starts at 23:00.. the next nearest bar is 00:00 the following day.

HTH 

 
  1. honest_knave: It appears to be correct. There isn't an H4 or D1 bar that starts at 23:00.. the next nearest bar is 00:00 the following day.

    Agreed. at the start of the tester you have a guaranteed 100 H1 bars and the contained lower bars, but not higher ones, they come from history.

    So iBarShift returns the earliest available index (next day) and likely sets the error 4066. By the time you get the next call, history has been updated and you get the correct 20:00

  2. if (counted_bars > 0) counted_bars--;
    No need for the decrement. Contradictory information on IndicatorCounted() - MQL4 forum That is why your indicator keeps looping 0,1,0,1...
  3. Stop using the old IndicatorCounted with the new OnCalculate. How to do your lookbacks correctly.
 
gxforex:

However when doing a back test it retrieved wrong values. 

Are you doing a backtest of the indicator, or are you doing a backtest of an EA and then manually adding the multi-timeframe indicator onto the backtesting chart for the EA?

The latter doesn't work properly in MT4. You can add indicators to the backtesting chart of an EA, but use of timeseries functions such as iBarShift and iClose within such a separate indicator breaks out of the backtesting sandbox.

For example, do a visual-mode backtest of any EA on a period such as H1, and then drop the following indicator onto the backtesting chart. It should display the latest simulated close price within the backtesting environment, just like it shows the live price on a live chart. But it doesn't. 

#property indicator_chart_window

int OnCalculate(const int rates_total, const int prev_calculated, const datetime &time[], const double &open[], const double &high[], 
const double &low[], const double &close[], const long &tick_volume[], const long &volume[], const int &spread[])
{
   // Shows the last real close price from the live market, before the backtesting starts.
   // Does NOT show the constantly changing latest price within the backtesting environment.
   Comment("Close price: ", iClose(Symbol(), PERIOD_D1, 0));
   return(rates_total);
}

Doing a backtest of this indicator, rather than doing a backtest of an EA and adding the indicator to the EA's chart, does work correctly.

 
honest_knave:

It appears to be correct.

There isn't an H4 or D1 bar that starts at 23:00.. the next nearest bar is 00:00 the following day.

HTH 

Hi honest_knave, thanks for the clarification. I would think it to be like, H4 2016.09.14 20:00. It is meaningless to find the next nearest bar in a future time... 
 
WHRoeder:
  1. honest_knave: It appears to be correct. There isn't an H4 or D1 bar that starts at 23:00.. the next nearest bar is 00:00 the following day.

    Agreed. at the start of the tester you have a guaranteed 100 H1 bars and the contained lower bars, but not higher ones, they come from history.

    So iBarShift returns the earliest available index (next day) and likely sets the error 4066. By the time you get the next call, history has been updated and you get the correct 20:00

  2. No need for the decrement. Contradictory information on IndicatorCounted() - MQL4 forum That is why your indicator keeps looping 0,1,0,1...
  3. Stop using the old IndicatorCounted with the new OnCalculate. How to do your lookbacks correctly.
Thank you WHRoeder, I have modified the code according to your examples. Please see the new posted
 
jjc:

Are you doing a backtest of the indicator, or are you doing a backtest of an EA and then manually adding the multi-timeframe indicator onto the backtesting chart for the EA?

The latter doesn't work properly in MT4. You can add indicators to the backtesting chart of an EA, but use of timeseries functions such as iBarShift and iClose within such a separate indicator breaks out of the backtesting sandbox.

For example, do a visual-mode backtest of any EA on a period such as H1, and then drop the following indicator onto the backtesting chart. It should display the latest simulated close price within the backtesting environment, just like it shows the live price on a live chart. But it doesn't. 

Doing a backtest of this indicator, rather than doing a backtest of an EA and adding the indicator to the EA's chart, does work correctly.

Hi jjc,

I dragged the indicator into the backtest window running an EA. The EA actually does nothing only for testing purpose.

I did a backtest of the indicator only and it seemed to be running ok. However I do need it to be added on an EA chart because I need to know whether an EA is correctly calling an indicator...

 

Here is the new code. I tested again on some other periods. I don't understand why H4, D1 and MN1 became some time two months before???


#property strict
#property indicator_chart_window

#define LAST 1
int lookback=1;

int OnInit(){   
   return(INIT_SUCCEEDED);
  }
//+------------------------------------------------------------------+
//| Custom indicator iteration function                              |
//+------------------------------------------------------------------+
int OnCalculate(const int rates_total, 
                const int prev_calculated, 
                const datetime& time[], 
                const double& open[], 
                const double& high[], 
                const double& low[], 
                const double& close[], 
                const long& tick_volume[], 
                const long& volume[], 
                const int& spread[]) {
                
   for(int bar = Bars-1-MathMax(lookback, prev_calculated); bar >= LAST; --bar){
      datetime dt=Time[bar];
      datetime Time_H1=iTime(Symbol(),PERIOD_H1,iBarShift(Symbol(),PERIOD_H1,dt));
      datetime Time_H4=iTime(Symbol(),PERIOD_H4,iBarShift(Symbol(),PERIOD_H4,dt));
      datetime Time_D1=iTime(Symbol(),PERIOD_D1,iBarShift(Symbol(),PERIOD_D1,dt));
      datetime Time_MN1=iTime(Symbol(),PERIOD_MN1,iBarShift(Symbol(),PERIOD_MN1,dt));
      Print(bar,": dt=",dt," Time_H1=",Time_H1," Time_H4=",Time_H4," Time_D1=",Time_D1," Time_MN1=",Time_MN1);
   }
   return rates_total-MathMax(1,LAST);
}


 
gxforex:

I don't understand why H4, D1 and MN1 became some time two months before???

As far as I am aware, it's because - like I said above - what you are trying to do just doesn't work properly in MT4. You can backtest a multi-timeframe indicator itself, i.e. selecting "Indicator" rather than "Expert Adviser" in the strategy tester, but you cannot successfully backtest an EA and then add the multi-timeframe indicator to the EA's chart. The timeseries functions such as iBarShift do not work properly in the separate indicator if you do this. They only work properly in the thing - the EA or indicator - actually being backtested by MT4.

(And, even if you attempt to fix things such as the iBarShift index, the separate indicator can then see into the future. For example, in the middle of a backtested simulated day, a separate indicator can see the future close price for that day, giving you information which you would not have had at the time, and usually making your indicator appear unrealistically successful and informative.)

 
jjc:

As far as I am aware, it's because - like I said above - what you are trying to do just doesn't work properly in MT4. You can backtest a multi-timeframe indicator itself, i.e. selecting "Indicator" rather than "Expert Adviser" in the strategy tester, but you cannot successfully backtest an EA and then add the multi-timeframe indicator to the EA's chart. The timeseries functions such as iBarShift do not work properly in the separate indicator if you do this. They only work properly in the thing - the EA or indicator - actually being backtested by MT4.

(And, even if you attempt to fix things such as the iBarShift index, the separate indicator can then see into the future. For example, in the middle of a backtested simulated day, a separate indicator can see the future close price for that day, giving you information which you would not have had at the time, and usually making your indicator appear unrealistically successful and informative.)

Thanks jjc for your explainations. So it will be difficult to test an EA which invokes other indicators, especially when multiple timeframs indicators involved.... :(

This should be a bug. I hope MetaTader can get it fixed in a short future. 

 
gxforex:

So it will be difficult to test an EA which invokes other indicators

It depends on the indicators. An indicator which only uses the values passed in the buffers to OnCalculate() should be fine. An indicator which tries to get other data using the timeseries functions iBarShift, iTime, iClose etc will not work properly.

The problem is specifically with multi-timeframe indicators. 

(Much the same applies to offline charts. A multi-timeframe indicator on an offline chart sees the standard bar history, not the offline chart history.)

12
New comment
Reason: