EA Buffers Not Getting Correct Values from Custom Indicator Buffer

 

Hi all,

I'm trying to make an EA and backtest it, but I'm having problems with the buffers in my EA. I know that in my custom indicator the values are getting set correctly in the buffers there as I have debugged through that and checked. However in my EA when I use CopyBuffer(), I am not getting the correct values. All I am getting is some random values . I even tried to figure this out using the most simplistic code of an indicator and an EA, which really do nothing but set buffer values in the indicator and copy the buffer values in the EA. So my question is how do I actually get buffer arrays from a custom indicator. Here's the code from my simplistic example as I think if I get the answer for this code, then I will be able to figure it out for my actual more complicated EA:

//+------------------------------------------------------------------+
//|                                                       TestEA.mq5 |
//|                        Copyright 2019, MetaQuotes Software Corp. |
//|                                             https://www.mql5.com |
//+------------------------------------------------------------------+
#property copyright "Copyright 2019, MetaQuotes Software Corp."
#property link      "https://www.mql5.com"
#property version   "1.00"
#property tester_indicator "TestIndicator.ex5"

double Test[];
int handle;
//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
  {
//--- create timer
   handle = iCustom(NULL, 0, "TestIndicator");
   CopyBuffer(handle, 0, 0, 100000, Test);
//---
   return(INIT_SUCCEEDED);
  }
//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
  {
//--- destroy timer
   IndicatorRelease(handle);
  }
//+------------------------------------------------------------------+
//|                                                TestIndicator.mq5 |
//|                        Copyright 2019, MetaQuotes Software Corp. |
//|                                             https://www.mql5.com |
//+------------------------------------------------------------------+
#property copyright "Copyright 2019, MetaQuotes Software Corp."
#property link      "https://www.mql5.com"
#property version   "1.00"
#property indicator_chart_window
#property indicator_buffers 1

double TestBuffer[];
int inserted = 0;
//+------------------------------------------------------------------+
//| Custom indicator initialization function                         |
//+------------------------------------------------------------------+
int OnInit()
  {
//--- indicator buffers mapping
   SetIndexBuffer(0, TestBuffer);
   IndicatorSetString(INDICATOR_SHORTNAME, "Testing");
//---
   return(INIT_SUCCEEDED);
  }
//+------------------------------------------------------------------+
//| Custom indicator iteration function                              |
//+------------------------------------------------------------------+
int OnCalculate(const int rates_total,
                const int prev_calculated,
                const int begin,
                const double &price[])
  {
//---
   TestBuffer[inserted] = inserted;
   ++inserted;
//--- return value of prev_calculated for next call
   return(rates_total);
  }
Attached is also an image of what I'm seeing when I debug if that helps at all. Test[2] always seems to be 0 for some reason. Any feedback is appreciated. Thank you.
Files:
 

Indicator:

//+------------------------------------------------------------------+
//|                                               Test_Indicator.mq5 |
//|                              Copyright © 2019, Vladimir Karputov |
//|                                           http://wmua.ru/slesar/ |
//+------------------------------------------------------------------+
#property copyright "Copyright © 2019, Vladimir Karputov"
#property link      "http://wmua.ru/slesar/"
#property version   "1.00"
#property indicator_separate_window
#property indicator_buffers 1
#property indicator_plots   1
//--- plot BarNumber
#property indicator_label1  "BarNumber"
#property indicator_type1   DRAW_LINE
#property indicator_color1  clrLimeGreen
#property indicator_style1  STYLE_SOLID
#property indicator_width1  1
//--- input parameters

//--- indicator buffers
double         BarNumberBuffer[];
//+------------------------------------------------------------------+
//| Custom indicator initialization function                         |
//+------------------------------------------------------------------+
int OnInit()
  {
//--- indicator buffers mapping
   SetIndexBuffer(0,BarNumberBuffer,INDICATOR_DATA);
//--- set the accuracy of values to be displayed in the Data Window
   IndicatorSetInteger(INDICATOR_DIGITS,0);
//---
   return(INIT_SUCCEEDED);
  }
//+------------------------------------------------------------------+
//| Custom indicator iteration function                              |
//+------------------------------------------------------------------+
int OnCalculate(const int rates_total,
                const int prev_calculated,
                const datetime &time[],
                const double &open[],
                const double &high[],
                const double &low[],
                const double &close[],
                const long &tick_volume[],
                const long &volume[],
                const int &spread[])
  {
//---
   int limit=prev_calculated-1;
   if(prev_calculated==0)
      limit=0;
   for(int i=limit; i<rates_total; i++)
     {
      BarNumberBuffer[i]=i;
     }
//--- return value of prev_calculated for next call
   return(rates_total);
  }
//+------------------------------------------------------------------+

Test_Indicator


EA:

//+------------------------------------------------------------------+
//|                                                      Test_EA.mq5 |
//|                              Copyright © 2019, Vladimir Karputov |
//|                                           http://wmua.ru/slesar/ |
//+------------------------------------------------------------------+
#property copyright "Copyright © 2019, Vladimir Karputov"
#property link      "http://wmua.ru/slesar/"
#property version   "1.00"
//--- input parameters

//---
int    handle_iCustom;                       // variable for storing the handle of the iCustom indicator
//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
  {
//--- create handle of the indicator iCustom
   handle_iCustom=iCustom(Symbol(),Period(),"Test_Indicator");
//--- if the handle is not created
   if(handle_iCustom==INVALID_HANDLE)
     {
      //--- tell about the failure and output the error code
      PrintFormat("Failed to create handle of the iCustom indicator for the symbol %s/%s, error code %d",
                  Symbol(),
                  EnumToString(Period()),
                  GetLastError());
      //--- the indicator is stopped early
      return(INIT_FAILED);
     }
//---
   return(INIT_SUCCEEDED);
  }
//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
  {
//---

  }
//+------------------------------------------------------------------+
//| Expert tick function                                             |
//+------------------------------------------------------------------+
void OnTick()
  {
//---
   double buffers[];
   ArraySetAsSeries(buffers,true);
   int start_pos=0,count=3;
   if(!iGetArray(handle_iCustom,0,start_pos,count,buffers))
      return;
//---
   string text="";
   for(int i=0; i<count; i++)
      text=text+"bar #"+IntegerToString(i)+": "+DoubleToString(buffers[i],0)+"\n";
   Comment(text);
  }
//+------------------------------------------------------------------+
//| Get value of buffers                                             |
//+------------------------------------------------------------------+
bool iGetArray(const int handle,const int buffer,const int start_pos,
               const int count,double &arr_buffer[])
  {
   bool result=true;
   if(!ArrayIsDynamic(arr_buffer))
     {
      PrintFormat("ERROR! EA: %s, FUNCTION: %s, this a no dynamic array!",__FILE__,__FUNCTION__);
      return(false);
     }
   ArrayFree(arr_buffer);
//--- reset error code
   ResetLastError();
//--- fill a part of the iBands array with values from the indicator buffer
   int copied=CopyBuffer(handle,buffer,start_pos,count,arr_buffer);
   if(copied!=count)
     {
      //--- if the copying fails, tell the error code
      PrintFormat("ERROR! EA: %s, FUNCTION: %s, amount to copy: %d, copied: %d, error code %d",
                  __FILE__,__FUNCTION__,count,copied,GetLastError());
      //--- quit with zero result - it means that the indicator is considered as not calculated
      return(false);
     }
   return(result);
  }
//+------------------------------------------------------------------+

Test_EA

Files:
 
HandsomePapito:

Hi all,

I'm trying to make an EA and backtest it, but I'm having problems with the buffers in my EA. I know that in my custom indicator the values are getting set correctly in the buffers there as I have debugged through that and checked. However in my EA when I use CopyBuffer(), I am not getting the correct values. All I am getting is some random values . I even tried to figure this out using the most simplistic code of an indicator and an EA, which really do nothing but set buffer values in the indicator and copy the buffer values in the EA. So my question is how do I actually get buffer arrays from a custom indicator. Here's the code from my simplistic example as I think if I get the answer for this code, then I will be able to figure it out for my actual more complicated EA:

Attached is also an image of what I'm seeing when I debug if that helps at all. Test[2] always seems to be 0 for some reason. Any feedback is appreciated. Thank you.

(updated) I came across almost the same situation as you.

The same Indicator and the same EA, exactly the same settings, run the same test multiple times, there is a high probability of getting randomly incorrect results.

If I initialize the output value of indicators with a value of zero in advance, most of the false test results are all zero and a few are correct.

I now believe that the iCustom() function of MT5/MQL5 is itself faulty.

Since the probability of errors in the optimization series of the same test is much higher than the probability of errors in the individual test, my unreliable guess is:

It may be related to the simultaneous invocation of historical data in multiple test processes.

I tried to completely port the algorithm in my custom indicator into EA, using the CopyRates() function called conditionally in OnTick() to compute the "indicator" value.

All the test results are exactly the same, and all the results are correct.
Documentation on MQL5: Timeseries and Indicators Access / CopyRates
Documentation on MQL5: Timeseries and Indicators Access / CopyRates
  • www.mql5.com
Gets history data of MqlRates structure of a specified symbol-period in specified quantity into the rates_array array. The elements ordering of the copied data is from present to the past, i.e., starting position of 0 means the current bar. If you know the amount of data you need to copy, it should better be done to a statically allocated...
 
tickfenix :


Before blaming the iCustom function, you first need to start with yourself: if your code is written carelessly, with system errors, then iCustom is not to blame. How much experience do I have that everyone who screams “iCustom is bad” actually writes their code like “chicken paw” (they write a disgusting code).

This is for information.


Let me remind you that this is a MQL5 technical forum and if you have a question, ask it. Provide a code. Describe the test conditions. Describe the result.

 
Vladimir Karputov:

Indicator:



EA:


Thanks for the response, Vladimir. I tried out your code and it worked. I'll have to take a closer look after work hopefully if I have time.
tickfenix:


I came across almost the same situation as you.

The same Indicator and the same EA, exactly the same settings, run the same test multiple times, there is a high probability of getting randomly incorrect results.

If I initialize the output value of indicators with a value of zero in advance, most of the false test results are all zero and a few are correct.

I now believe that the iCustom() function of MT5/MQL5 is itself faulty.

Since the probability of errors in the optimization series of the same test is much higher than the probability of errors in the individual test, my unreliable guess is:

It may be related to the simultaneous invocation of historical data in multiple test processes.

I tried to completely port the algorithm in my custom indicator into EA, using the CopyRates() function called conditionally in OnTick() to compute the "indicator" value.

All the test results are exactly the same, and all the results are correct.
That's interesting to hear, tickfenix. It would seem that my test indicator and EA should have worked and I should have been able to see the values in correctly in my EA, so it is very odd. I'll compare Vladimir's code more closely if I can after work hopefully or whenever I get the chance.
 
HandsomePapito :
Thanks for the response, Vladimir. I tried out your code and it worked. I'll have to take a closer look after work hopefully if I have time.

Please note: first I apply to arrays ArraySetAsSeries

   double buffers[];
   ArraySetAsSeries(buffers,true);

and copy only the last three bars

   int start_pos=0,count=3;
   if(!iGetArray(handle_iCustom,0,start_pos,count,buffers))
      return;


Nothing complicated. Everything is very simple!

Documentation on MQL5: Array Functions / ArraySetAsSeries
Documentation on MQL5: Array Functions / ArraySetAsSeries
  • www.mql5.com
//| Custom indicator initialization function                         | //| Custom indicator iteration function                              |
 
HandsomePapito:

Hi all,

I'm trying to make an EA and backtest it, but I'm having problems with the buffers in my EA. I know that in my custom indicator the values are getting set correctly in the buffers there as I have debugged through that and checked. However in my EA when I use CopyBuffer(), I am not getting the correct values. All I am getting is some random values . I even tried to figure this out using the most simplistic code of an indicator and an EA, which really do nothing but set buffer values in the indicator and copy the buffer values in the EA. So my question is how do I actually get buffer arrays from a custom indicator. Here's the code from my simplistic example as I think if I get the answer for this code, then I will be able to figure it out for my actual more complicated EA:

Attached is also an image of what I'm seeing when I debug if that helps at all. Test[2] always seems to be 0 for some reason. Any feedback is appreciated. Thank you.


error 1.

in your indicator:

TestBuffer[inserted] = inserted;

The index number of TestBuffer in you indicator should always be a bar number like 350188,depending on the history data your test running on,it should never be a number defined by you.

try:

TestBuffer[rates_total-1]=inserted;


error 2.

in your ea:

indicator call in OnInit() means your indicator will be called only once at the first bar.

no further accumulation of variable "inserted", only zero.

try putting CopyBuffer() into OnTick().

thus:

ticks flow, then calculation happens on corresponding bar. till the very last second of your test process.


3. not an error. only an option.

set your receiving array Test[] to timeseries if you want Test[0] to have the most recent indicator value:

ArraySetAsSeries(Test,true); 

The order in which you can manipulate the data you extract from the indicator is entirely up to you.


4. 

I modified your test indicator and ea to make them work. 

You can try it and see if the output is what you need.

please observe the output number after " ------ ". it's the index number of the indicator buffer , a number you can not specify.

//+------------------------------------------------------------------+
//|                                                TestIndicator.mq5 |
//|                        Copyright 2019, MetaQuotes Software Corp. |
//|                                             https://www.mql5.com |
//+------------------------------------------------------------------+
#property copyright "Copyright 2019, MetaQuotes Software Corp."
#property link      "https://www.mql5.com"
#property version   "1.00"
#property indicator_chart_window
#property indicator_buffers 1

double TestBuffer[];
int inserted = 0;
//+------------------------------------------------------------------+
//| Custom indicator initialization function                         |
//+------------------------------------------------------------------+
int OnInit()  
{
   SetIndexBuffer(0,TestBuffer,INDICATOR_DATA);
   IndicatorSetString(INDICATOR_SHORTNAME, "Testing");
   return(INIT_SUCCEEDED); 
}
//+------------------------------------------------------------------+
//| Custom indicator iteration function                              |
//+------------------------------------------------------------------+
int OnCalculate(const int rates_total,
                const int prev_calculated,
                const int begin,
                const double &price[])  
{
   TestBuffer[rates_total-1] = inserted;
   Print(" ------ ",rates_total-1,":  ",DoubleToString(TestBuffer[rates_total-1],0));
   inserted++;
return(rates_total);
}
//+------------------------------------------------------------------+
//+------------------------------------------------------------------+
//|                                                       TestEA.mq5 |
//|                        Copyright 2019, MetaQuotes Software Corp. |
//|                                             https://www.mql5.com |
//+------------------------------------------------------------------+
#property copyright "Copyright 2019, MetaQuotes Software Corp."
#property link      "https://www.mql5.com"
#property version   "1.00"
#property tester_indicator "TestIndicator.ex5"

double Test[];
int handle;
int couter=2000;
//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
  {
//   ArraySetAsSeries(Test,true);
   handle = iCustom(NULL, 0, "TestIndicator");
   if ( handle==INVALID_HANDLE ) { Print("Failed to get IndicatorHandle!"); ExpertRemove(); }
   return(INIT_SUCCEEDED);
  }
//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
  {
   IndicatorRelease(handle);
  }
//+------------------------------------------------------------------+
void OnTick()
{

   CopyBuffer(handle, 0, 0, 3, Test);
   for ( int i=0;i<3;i++ ) Print(i,"  ",DoubleToString(Test[i],0));
   Print("");
   couter--;
   if ( couter==0 ) TesterStop();
   
return;
}
 

Wait for the next bar, than you habe 1 more

you count the existing bars in the chart

 
Vladimir Karputov:

Please note: first I apply to arrays ArraySetAsSeries

and copy only the last three bars


Nothing complicated. Everything is very simple!

Thanks for pointing that out, Vladimir. I have an opportunity to take a better look now, so I'll look closely at those points.
tickfenix:


error 1.

in your indicator:

The index number of TestBuffer in you indicator should always be a bar number like 350188,depending on the history data your test running on,it should never be a number defined by you.

try:


error 2.

in your ea:

indicator call in OnInit() means your indicator will be called only once at the first bar.

no further accumulation of variable "inserted", only zero.

try putting CopyBuffer() into OnTick().

thus:

ticks flow, then calculation happens on corresponding bar. till the very last second of your test process.


3. not an error. only an option.

set your receiving array Test[] to timeseries if you want Test[0] to have the most recent indicator value:

The order in which you can manipulate the data you extract from the indicator is entirely up to you.


4. 

I modified your test indicator and ea to make them work. 

You can try it and see if the output is what you need.

please observe the output number after " ------ ". it's the index number of the indicator buffer , a number you can not specify.

Thanks for all the tips here. As for #2, that part was on purpose. I simply wanted to see the indicator buffers copy data correctly. I'll see if your updated code works for me. Thanks for taking the time to do that.

 
amando:

Wait for the next bar, than you habe 1 more

you count the existing bars in the chart

Yeah, I realize it's better to wait for the next bar and keep the data in the buffers aligned with the data on your chart. I was just meaning this example as the most simple thing I could come up with to try and see it work. So for real implementations, I'll be sure to keep that in mind. Thanks.
 
I have difficulties with getting simple moving average values in a custom indicator. It should be simple (pun intended) but I can't get it right for several hours now.
All I need are 10 bars of data but instead of the right outcome I have
  • no data at all
  • all data are the same (not acceptable, I need the proper MA values)
  • array out of range error messages

I thought I've finally fixed these issues but it turned out my solution isn't correct because sometimes the data can change in previous bars - which cannot happen with moving averages. 

Here is my code:

double ExtBuffer[];
double dMABuffer[];

//+------------------------------------------------------------------+
//| Custom indicator initialization function                         |
//+------------------------------------------------------------------+
int OnInit()
{
//--- indicator buffers mapping
        SetIndexBuffer(0, ExtBuffer, INDICATOR_DATA);
//--- indicator accuracy to be displayed 
        IndicatorSetInteger(INDICATOR_DIGITS, _Digits);
//---- indicator name to be displayed in DataWindow and subwindow 
        IndicatorSetString(INDICATOR_SHORTNAME, "Trend");
//--- set index of the bar the drawing starts from 
        //PlotIndexSetInteger(0, PLOT_DRAW_BEGIN, 10);
        PlotIndexSetInteger(0, PLOT_LINE_STYLE, STYLE_SOLID);
//--- sets drawing line empty value
        PlotIndexSetDouble(0, PLOT_EMPTY_VALUE, 0.0);
//---
        iMAHandle = iMA(NULL, PERIOD_CURRENT, 10, 0, MODE_SMA, PRICE_CLOSE);
        if ( iMAHandle == INVALID_HANDLE )
        {
                Print(__FUNCTION__, " MA handle invalid at init!");
                return (INIT_FAILED);
        }
        ArraySetAsSeries(dMABuffer, true);
}

//+------------------------------------------------------------------+
//| Custom indicator iteration function                              |
//+------------------------------------------------------------------+
int OnCalculate(const int rates_total,
                const int prev_calculated,
                const datetime &Time[],
                const double &Open[],
                const double &High[],
                const double &Low[],
                const double &Close[],
                const long &tick_volume[],
                const long &volume[],
                const int &spread[])
{
        int i, limit, j, j1 = 0, cnt = 0;
        
        bool bEnd = false;
        double d;
//--- preliminary calculations
        if ( !prev_calculated )
        {
                //--- set first data
                ExtBuffer[0] = Close[0];
                limit = 1;
        }
        else limit = prev_calculated - 1;

//--- the main loop of calculations
        for ( i = limit; i < rates_total && !IsStopped(); i++ )
        {
                if ( i < rates_total - 10 )
                {
                        if ( CopyBuffer(iMAHandle, 0, i, 10, dMABuffer) < 0 )           //99% of time this is fine
                        {
                                //Print(__FUNCTION__, ", Error copying MA indicator buffer! ");
                                return -1;
                        }
                }
                else
                {
                        if ( (cnt = CopyBuffer(iMAHandle, 0, i, rates_total - i, dMABuffer)) < 0 )
                        {
                                Print(__FUNCTION__, ", Error copying MA indicator buffer! ");
                                return -1;
                        }
                        if ( !bEnd )
                        {
                                bEnd = true;j1 = 0;
                        }
                }
                if ( !End )
                        d = dMABuffer[0] - dMABuffer[1];
                if ( bEnd )                             //no matter what I do, if this condition is met the buffer is filled with garbage.
                ;                                               //Even if I explicitly check for the cnt variable to see how much data has been copied - it still doesn't work.
        }
}

If I use this approach at the end

if ( (cnt = CopyBuffer(iMAHandle, 0, 0, rates_total - 1, dMABuffer)) < 0 )

then the data seems to be good. Calculations work. I just need to increase the indexing of dMABuffer. There is only one problem: MA values can change over time in previous bars. I don't think this should happen. Any help would be appreciated.

Reason: