Making a faster correlation indicator, using ALGLIB

oneilljo 2014.02.17 05:03

Hello all. I am trying to make an indicator that shows the Pearson product moment correlation between 2 currency pairs, using the ALGLIB library. There is already a correlation indicator in the codebase, but it is very slow.

This is what I have so far:

//+------------------------------------------------------------------+
//|                                                  Correlation.mq5 |
//|                        Copyright 2009, MetaQuotes Software Corp. |
//|                                              http://www.mql5.com |
//+------------------------------------------------------------------+
#property copyright "2009, MetaQuotes Software Corp."
#property link      "http://www.mql5.com"
#property version   "2.00"
#property indicator_separate_window
#property indicator_buffers 2
#property indicator_plots   1
#property indicator_applied_price PRICE_CLOSE
#include <Math\Alglib\alglib.mqh>
//---- plot CORRELATION
#property indicator_label1  "CORR"
#property indicator_type1   DRAW_LINE
#property indicator_color1  Blue
#property indicator_style1  STYLE_SOLID
#property indicator_width1  2
#property indicator_minimum -1
#property indicator_maximum 1
//--- input parameters
input string  secondSymbol="EURUSD";  // Second symbol
input int     setPeriod=1440; 
//--- indicator buffers and handles
double         CORRBuffer[];
double         TempDataBuffer1[];
int            ExtHandle=0;

//+------------------------------------------------------------------+
//| Custom indicator initialization function                         |
//+------------------------------------------------------------------+
int OnInit()
  {
//--- indicator buffers mapping
   SetIndexBuffer(0,CORRBuffer,INDICATOR_DATA);
   SetIndexBuffer(1,TempDataBuffer1,INDICATOR_CALCULATIONS);
//--- bar, starting from which the indicator is drawn
   PlotIndexSetInteger(0,PLOT_DRAW_BEGIN,setPeriod);
   string shortname;
   StringConcatenate(shortname,"CORR(",secondSymbol,",",setPeriod,")");
//--- set a label do display in DataWindow
   PlotIndexSetString(0,PLOT_LABEL,shortname);   
//--- set a name to show in a separate sub-window or a pop-up help
   IndicatorSetString(INDICATOR_SHORTNAME,shortname);
//--- set accuracy of displaying the indicator values
   IndicatorSetInteger(INDICATOR_DIGITS,2);
//--- 
   ExtHandle=iMA(secondSymbol,0,1,0,MODE_SMA,PRICE_CLOSE);
   if(ExtHandle==INVALID_HANDLE)
     {
      printf("Error creating MA indicator");
      return(INIT_FAILED);
     }
   return(0);
  }
//+------------------------------------------------------------------+
//| Custom indicator iteration function                              |
//+------------------------------------------------------------------+
int OnCalculate (const int rates_total,    // size of the price[] array;
                 const int prev_calculated,// number of available bars;
                 // at the previous call;
                 const int begin,// from what index of the 
                 // price[] array true data start;
                 const double &price[]) // array, at which the indicator will be calculated;
  {
//--- get price array for second currency pair, same length as &price[] array  
   MqlRates rt[];
   CopyRates(secondSymbol,_Period,0,rates_total,rt);
//--- fill calculation array
   for(int i=prev_calculated;i<rates_total;i++)
     {
      TempDataBuffer1[i]=rt[i].close;
     }
//--- calculate correlation
    double TempDataBuffer2[];
    double TempDataBuffer3[];
    for(int i=prev_calculated+setPeriod;i<rates_total;i++)
      {
       if(ArrayCopy(TempDataBuffer2,TempDataBuffer1,0,i-setPeriod,setPeriod)!=setPeriod)
         {
          Print("error in the array copying");
          return(rates_total);
         } 
       if(ArrayCopy(TempDataBuffer3,price,0,i-setPeriod,setPeriod)!=setPeriod)
         {
          Print("error in the array copying");
          return(rates_total);
         }  
       CORRBuffer[i]=CAlglib::PearsonCorr2(TempDataBuffer2,TempDataBuffer3);
       ArrayFree(TempDataBuffer2);
       ArrayFree(TempDataBuffer3);
      } 
//--- return   
   return(rates_total);

  }

The indicator works as expected when first attached to the chart. However, when a new bar comes in, the correlation indicator always shows a zero value for the new bar. (See screenshot below).

Does anyone have any ideas on how to fix this.... I am stumped

oneillj 2014.02.18 09:08 #1

Fixed. The indicator works perfectly on most symbols, but for some reason on AUDUSD chart, CopyBuffer() fails and the error message is printed. The indicator won't work only on the AUDUSD chart.

Does anyone have any idea why that might be?

Note that the CopyBuffer() function is called to copy indicator data from an iMA indicator applied to the EURUSD symbol every time (the second symbol for correlation calculations) . So it should work regardless of what chart this is attached to. Why does it fail just for the AUDUSD chart?

//+------------------------------------------------------------------+
//|                                                  Correlation.mq5 |
//|                        Copyright 2009, MetaQuotes Software Corp. |
//|                                              http://www.mql5.com |
//+------------------------------------------------------------------+
#property copyright "2009, MetaQuotes Software Corp."
#property link      "http://www.mql5.com"
#property version   "2.00"
#property indicator_separate_window
#property indicator_buffers 2
#property indicator_plots   1
#property indicator_applied_price PRICE_CLOSE
#include <Math\Alglib\alglib.mqh>
//---- plot CORRELATION
#property indicator_label1  "CORR"
#property indicator_type1   DRAW_LINE
#property indicator_color1  Blue
#property indicator_style1  STYLE_SOLID
#property indicator_width1  2
#property indicator_minimum -1
#property indicator_maximum 1
//--- input parameters
input string  secondSymbol="EURUSD";  // Second symbol
input int     setPeriod=1440; 
//--- indicator buffers and handles
double         CORRBuffer[];
double         TempDataBuffer1[];
int            ExtHandle=0;

//+------------------------------------------------------------------+
//| Custom indicator initialization function                         |
//+------------------------------------------------------------------+
int OnInit()
  {
//--- indicator buffers mapping
   SetIndexBuffer(0,CORRBuffer,INDICATOR_DATA);
   SetIndexBuffer(1,TempDataBuffer1,INDICATOR_CALCULATIONS);
//--- bar, starting from which the indicator is drawn
   PlotIndexSetInteger(0,PLOT_DRAW_BEGIN,setPeriod);
   string shortname;
   StringConcatenate(shortname,"CORR(",secondSymbol,",",setPeriod,")");
//--- set a label do display in DataWindow
   PlotIndexSetString(0,PLOT_LABEL,shortname);   
//--- set a name to show in a separate sub-window or a pop-up help
   IndicatorSetString(INDICATOR_SHORTNAME,shortname);
//--- set accuracy of displaying the indicator values
   IndicatorSetInteger(INDICATOR_DIGITS,2);
//--- 
   ExtHandle=iMA(secondSymbol,0,1,0,MODE_SMA,PRICE_CLOSE);
   if(ExtHandle==INVALID_HANDLE)
     {
      printf("Error creating MA indicator");
      return(INIT_FAILED);
     }
   return(0);
  }
//+------------------------------------------------------------------+
//| Custom indicator iteration function                              |
//+------------------------------------------------------------------+
int OnCalculate (const int rates_total,    // size of the price[] array;
                 const int prev_calculated,// number of available bars;
                 // at the previous call;
                 const int begin,// from what index of the 
                 // price[] array true data start;
                 const double &price[]) // array, at which the indicator will be calculated;
  { 
//--- get price array for second currency pair, same length as &price[] array 
   if(CopyBuffer(ExtHandle,0,0,rates_total,TempDataBuffer1)!=rates_total)
     {
      Print("CopyBuffer failed, no data in second symbol price array");
      return 0;
     }
     
//--- calculate correlation
    double TempDataBuffer2[];
    double TempDataBuffer3[];
    int start;
    if(prev_calculated==0) start=setPeriod; // set the starting index for input arrays
    else start=prev_calculated-1;    // set 'start' equal to the last index in the arrays
    for(int i=start;i<rates_total;i++)
      {
       //CopyBuffer(ExtHandle,0,);
       ArrayCopy(TempDataBuffer2,TempDataBuffer1,0,i-setPeriod,setPeriod);
       ArrayCopy(TempDataBuffer3,price,0,i-setPeriod,setPeriod);  
       CORRBuffer[i]=CAlglib::PearsonCorr2(TempDataBuffer2,TempDataBuffer3);
       ArrayFree(TempDataBuffer2);
       ArrayFree(TempDataBuffer3);
      }     
//--- return   
   return(rates_total);

  }

VPS Failed to copy Questions in understanding the CopyBuffer error MQL5

Paul Anscombe 2014.02.18 09:36 #2

oneillj:

Fixed. The indicator works perfectly on most symbols, but for some reason on AUDUSD chart, CopyBuffer() fails and the error message is printed. The indicator won't work only on the AUDUSD chart.

Does anyone have any idea why that might be?

so what is the error message and have you tried accessing the last error details in your code?

oneilljo 2014.02.19 08:25 #3

The CopyBuffer() function fails, there is no error code. I tried printing an error code with Print(GetLastError());.

When it fails, the number of elements copied is one less than the amount specified in the function parameters. Or in the code below, "copied" is one less than rates_total. This causes the indicator not to work.

//--- get price array for second currency pair, same length as &price[] array 
   int copy=CopyBuffer(ExtHandle,0,0,rates_total,TempDataBuffer1);
   if(copy!=rates_total)
     {
      Print("copied ", copy);
      Print("rates_total ", rates_total);
      Print("CopyBuffer failed, no data in second symbol price array");
      return 0;
     }

I have been trying the indicator on various charts at various times. Sometimes the indicator works perfectly, other times not at all b/c CopyBuffer() is failing. Is this a MT5 thing?

Any ideas?

Mods: Should I move this to another section like "Technical indicators?"

Failed to copy data Help explaning the result CopyBuffer error for other

Alain Verleyen 2014.02.23 20:36 #4

oneilljo:

The CopyBuffer() function fails, there is no error code. I tried printing an error code with Print(GetLastError());.

I have been trying the indicator on various charts at various times. Sometimes the indicator works perfectly, other times not at all b/c CopyBuffer() is failing. Is this a MT5 thing?

Any ideas?

Mods: Should I move this to another section like "Technical indicators?"

We can't move a topic from one section to an other, don't worry about that.

The problem with CopyBuffer() is that you are trying to get "rates_total" value for your second symbol, but rates_total applied on current symbol/current timeframe, how do you know there is enough data in history for the second symbol ?

You have to use Bars(), see documentation.

Any rookie question, so Cannot Declare Array Pearson correlation

New comment