Self-regressive Median Coefficient - page 5

To add comments, please log in or register
Arthur Albano
341
Arthur Albano  

Fixed the error. MQL does not like swaping of two number that occupy the same position on an array. It writes ZERO. Now Median is faster, I believe, as it is quickselect, and it's worst case scenario is quicksort, the method used with ArraySort(). With help from https://www.mql5.com/en/users/fxsaber https://www.mql5.com/en/forum/275881/page2

#define SWAP(a, b) { if (a != b) a ^= b ^= a ^= b; }
#define SORT(a, b, c) {if(a > b) SWAP(a,b); if(a > c) SWAP(a,c); if (b>c) SWAP(b,c)}
// Partition using Lomuto partition scheme

int partition(int &list[],int left,int right,int pivotIndex)
  {
   int pivotValue = list[pivotIndex];
   SWAP(list[pivotIndex],list[right]);
   int storeIndex = left;
   for(int i=left; i<right; i++)
      {
         if(list[i] < pivotValue)
            {
               SWAP(list[storeIndex],list[i]);
               storeIndex++;
            }
      }
   SWAP(list[right],list[storeIndex]);

   return storeIndex;

  }

int quickselect(int &list[], int left, int right, int k)

  {
   if(left==right)
      return list[left];

   int pivotIndex=left+rand()%(right-left+1);

   pivotIndex=partition(list,left,right,pivotIndex);

// The pivot is in its final sorted position
   if(k==pivotIndex)
      return list[k];
   else if(k<pivotIndex)
      return quickselect(list, left, pivotIndex - 1, k);
   else
      return quickselect(list, pivotIndex + 1, right, k);

  }
//+------------------------------------------------------------------+
//|                                                                  |
//+------------------------------------------------------------------+
void OnStart()
  {

   int A[]={ 7,4,6,3,9,1 };

   int size=ArraySize(A);
   int medianRank=(size-1)/2;

   int med = (size%2==1)?quickselect(A,0,size-1,medianRank):(quickselect(A,0,size-1,medianRank)+quickselect(A,0,size-1,medianRank+1))/2;

   return;
  }
//+------------------------------------------------------------------+

Arthur Albano
341
Arthur Albano  
Alain Verleyen:

So at my surprise, I didn't succeed. The main optimization idea I had was to reduce the number of division operations needed :

As this operation is done in a loop for every candle i, there are a lot of repetitions of the exact same operation. So the idea was to do all the operations once and to memorize them (see attached how I did). However it doesn't improve this speed, even while the numbers of operations was reduced by a factor 16 !

From 64 millions to 4 millions division operations, but no change in execution time. I didn't expect that. That means double arithmetic CPU is very efficient and cached very well all the results.

Also, though this imbricated loop with division operations is time consuming, the main bottleneck in the ArraySort(), the speed impact is more than 3 times the one of the loops. So even if the "division" optimization had worked that global impact would have been low (~20% max).

That was an interesting exercise, even if it failed.

Attached the code (as we are on week-end I didn't pay attention to "live" update).

MLaden's code performance with ArraySort()


MLadens's code using quickselect for median:


So, if the newbie here doing nothing (much!) wrong, it seems quickselect is loosing to ArraySort(). My bad...

Alain Verleyen
36574
Alain Verleyen  
Arthur Albano:

MLaden's code performance with ArraySort()


MLadens's code using quickselect for median:


So, if the newbie here doing nothing (much!) wrong, it seems quickselect is loosing to ArraySort(). My bad...

There is very little chance to have mql code to be faster than ArraySort(), even with a better algorithm. MQL5 is a managed language, dynamic arrays indexing needs to be checked at run time. You would need to use static array with constant indexing to hope for an improvement, which is not really useful for most usage.

Arthur Albano
341
Arthur Albano  
Alain Verleyen:

There is very little chance to have mql code to be faster than ArraySort(), even with a better algorithm. MQL5 is a managed language, dynamic arrays indexing needs to be checked at run time. You would need to use static array with constant indexing to hope for an improvement, which is not really useful for most usage.

I hoped for more slicing, picking and copying in arrays, such as object oriented programming languages, such as Array.Sort(), Array.Median(), Array.Percent(PercentRank, C={0..1}), etc. Hard-coding selection in unsorted arrays should be faster than sorting and selecting, specially because of medians of median. And implementing a med(array) should be done using two-numbers selection algorithm, which is also faster than averaging two selections. But that's for MQL5 developers.

Arthur Albano
341
Arthur Albano  

HI all!

I have news! I will be traveling for a week and away from my laptop. So, enough of newbie code for you! :) But I already coded a percentile value based on percent rank. Bu adding the coefficients to price, I have a price projection. But also, the next logical step is to calculate Bowley's (Interquartile) Skewness, which is non-dimensional, but only uses numbers and no model equation (not Pearson), to compare different equities.


//------------------------------------------------------------------
#property copyright   "mladen"
#property link        "mladenfx@gmail.com"
//------------------------------------------------------------------

double QNaN   =(double)"nan";   // QNaN


#property indicator_chart_window
#property indicator_buffers 4
#property indicator_plots   3
#property indicator_label1  "Test value"
#property indicator_type1   DRAW_LINE
#property indicator_color1  clrAquamarine
#property indicator_type2   DRAW_LINE
#property indicator_color2  clrHotPink
#property indicator_type3   DRAW_LINE
#property indicator_color3  clrDarkKhaki

input int                inpPeriod = 15;          // Period
input ENUM_APPLIED_PRICE inpPrice  = PRICE_CLOSE; // Price
input int                inpShift  = 5;           // Shift
input double             inpPercentRankRange = 0.90; // Percent Rank Range [0..1]
double med[],max[],min[],prices[];
const double GoldenNumber = 1.61803399;
//------------------------------------------------------------------
//
//------------------------------------------------------------------
int OnInit()
{
   IndicatorSetInteger(INDICATOR_DIGITS,_Digits);

   SetIndexBuffer(0,med,INDICATOR_DATA);
   SetIndexBuffer(1,max,INDICATOR_DATA);
   SetIndexBuffer(2,min,INDICATOR_DATA);
   SetIndexBuffer(3,prices,INDICATOR_CALCULATIONS);

   PlotIndexSetInteger(0,PLOT_SHIFT,inpShift);
   PlotIndexSetInteger(1,PLOT_SHIFT,inpShift);
   PlotIndexSetInteger(2,PLOT_SHIFT,inpShift);

   PlotIndexSetInteger(0,PLOT_SHOW_DATA,false);
   IndicatorSetString(INDICATOR_SHORTNAME,"QuantileBands ("+(string)inpPeriod+")");
   return(INIT_SUCCEEDED);
}
void OnDeinit(const int reason) { return; }

//------------------------------------------------------------------
//
//------------------------------------------------------------------
int OnCalculate(const int rates_total,
                const int prev_calculated,
                const datetime& time[],
                const double& open[],
                const double& high[],
                const double& low[],
                const double& close[],
                const long& tick_volume[],
                const long& volume[],
                const int& spread[])
{                
   if (Bars(_Symbol,_Period)<rates_total) return(0);
   
   //
   //---
   //

   for (int i=(int)MathMax(prev_calculated-1,0); i<rates_total && !_StopFlag; i++)
   {
      prices[i] = getPrice(inpPrice,open,close,high,low,i,rates_total); 
      med[i] = prices[i]+MathQuantRegCoeff(i,inpPeriod,time,prices,0.5)*double(inpShift)*double(PeriodSeconds(PERIOD_CURRENT));
      min[i] = prices[i]+GoldenNumber*MathQuantRegCoeff(i,inpPeriod,time,prices,1-inpPercentRankRange)*double(inpShift)*double(PeriodSeconds(PERIOD_CURRENT));
      max[i] = prices[i]+GoldenNumber*MathQuantRegCoeff(i,inpPeriod,time,prices,inpPercentRankRange)*double(inpShift)*double(PeriodSeconds(PERIOD_CURRENT));
   }                  
   return(rates_total);
}


//------------------------------------------------------------------
//
//------------------------------------------------------------------
double CoeffBuffer[]; 
double MathQuantRegCoeff(int i, int _period, const datetime &time[], double &price[], double percent_rank)
{
   if (i<_period) return(0);
   int coeffSize = (_period*(_period-1))/2; if (ArraySize(CoeffBuffer)!=coeffSize) ArrayResize(CoeffBuffer,coeffSize); 
    
   for(int l=0,k=0; l<(_period-1); l++)
   for(int j=(l+1); j<(_period  ); j++,k++) CoeffBuffer[k] = (price[i-j]-price[i-l])/(time[i-j]-time[i-l]); ArraySort(CoeffBuffer);
                              
//   double  QuantRegCoeff = (coeffSize%2==1) ? (CoeffBuffer[coeffSize/2]) : (0.5*(CoeffBuffer[(coeffSize-1)/2]+CoeffBuffer[(coeffSize+1)/2]));
   double QuantRegCoeff = MathPercentileValue(CoeffBuffer, percent_rank, 0);
   return( QuantRegCoeff );
}

//+------------------------------------------------------------------+
//| Computes the interpolated percentile value of an array[]         |
//| todo: check for Mendenhall & Sicich (C=0)                        |
//+------------------------------------------------------------------+
double MathPercentileValue(double &array[], const double &percent_rank, const int C=0)
  {
//---
    double percentile_value;

//---
    int size=ArraySize(array);

//--- check data range
    if(size==0)
        return(QNaN);

//--- check for percent rank [0..1]
    if(percent_rank<0 || percent_rank>1)
        return(QNaN);
        
//--- simple case: one element
    if(size==1)
        return(array[0]);

   double rank;
   rank = (size - 1) * percent_rank;
   
   int left_rank;
   left_rank = (int)ceil(rank);
   double left_weight;
   left_weight = (rank + 1 ) - double(left_rank);
   
   int right_rank;
   right_rank = (int)floor(rank);
   double right_weight;
   right_weight = 1 - left_weight;


//--- prepare sorted values
   double sorted_values[];
   ArrayResize(sorted_values,size);
   if(ArrayCopy(sorted_values,array,0,0,WHOLE_ARRAY)!=size)
      return(QNaN);
   ArraySort(sorted_values);


   //--- calculation
    percentile_value = sorted_values[left_rank]*left_weight + sorted_values[right_rank]*right_weight;
   
   return(percentile_value);
}



//
//---
//    
double getPrice(ENUM_APPLIED_PRICE tprice,const double &open[],const double &close[],const double &high[],const double &low[],int i,int _bars)
{
   switch(tprice)
     {
      case PRICE_CLOSE:     return(close[i]);
      case PRICE_OPEN:      return(open[i]);
      case PRICE_HIGH:      return(high[i]);
      case PRICE_LOW:       return(low[i]);
      case PRICE_MEDIAN:    return((high[i]+low[i])/2.0);
      case PRICE_TYPICAL:   return((high[i]+low[i]+close[i])/3.0);
      case PRICE_WEIGHTED:  return((high[i]+low[i]+close[i]+close[i])/4.0);
     }
   return(0);
  }
Arthur Albano
341
Arthur Albano  

Hi people!


After a week at the ocens, I am back!

So, I have news. I decided do calculate the distance from a value to an underlying straight line. So, I did it in a few steps:

1) Use MLaden code;

2) Calculate the Intercept for that line;

3) Calculate the shortest distance (perhaps could be optmized for less loops);

4) Divide it by the last price (I have not decided my mind on last price or the point at the median line);

5) Divide by 3600 (seconds -> minutes);

6) Multiply by 100 to get percent.

I had to change the colors to mach SELL/BUY buttons at MT5. I miss these buttons so much. Sorry Mladen :)

I am not sure also if I get the distance at current time or the perpendicular distance (like a spread), in step 3.

But what is really anoyng me is to get time wright.

So, I have a problem getting distance for any period (step 5). The distance should not change too much, but it seems to change more than it should by changing time period. I have tried using PeriodSeconds(), PERIOD_CURRENT and _Period, but it didn't work as expected.

There goes the code:


//------------------------------------------------------------------
#property copyright   "mladen"
#property link        "mladenfx@gmail.com"
//------------------------------------------------------------------
#property indicator_separate_window
#property indicator_buffers 8
#property indicator_plots   2
#property indicator_label1  "Distance"
#property indicator_type1   DRAW_FILLING
#property indicator_color1  clrNavy,clrCrimson
#property indicator_label2  "Distance"
#property indicator_type2   DRAW_COLOR_LINE
#property indicator_color2  clrBlueViolet,clrLightPink
#property indicator_width2  2


input int                inpPeriod = 15;          // Period
input ENUM_APPLIED_PRICE inpPrice  = PRICE_CLOSE; // Price

double distance[],k0[],k1[],distancecolor[],prices[],fillu[],filld[];
//------------------------------------------------------------------
//
//------------------------------------------------------------------
int OnInit()
{
   IndicatorSetInteger(INDICATOR_DIGITS,2);
   SetIndexBuffer(0,fillu,INDICATOR_DATA);
   SetIndexBuffer(1,filld,INDICATOR_DATA);
   SetIndexBuffer(2,distance,INDICATOR_DATA);
   SetIndexBuffer(3,distancecolor,INDICATOR_COLOR_INDEX);
   SetIndexBuffer(4,prices,INDICATOR_CALCULATIONS);
   SetIndexBuffer(5,k0,INDICATOR_CALCULATIONS);
   SetIndexBuffer(6,k1,INDICATOR_CALCULATIONS);
   
   PlotIndexSetInteger(0,PLOT_SHOW_DATA,false);
   IndicatorSetString(INDICATOR_SHORTNAME,"Relative Perpendicular Distance ("+(string)inpPeriod+")");
   return(INIT_SUCCEEDED);
}
void OnDeinit(const int reason) { return; }

//------------------------------------------------------------------
//
//------------------------------------------------------------------
int OnCalculate(const int rates_total,
                const int prev_calculated,
                const datetime& time[],
                const double& open[],
                const double& high[],
                const double& low[],
                const double& close[],
                const long& tick_volume[],
                const long& volume[],
                const int& spread[])
{                
   if (Bars(_Symbol,_Period)<rates_total) return(0);
   
   //
   //---
   //

   for (int i=(int)MathMax(prev_calculated-1,0); i<rates_total && !_StopFlag; i++)
   {
      prices[i] = getPrice(inpPrice,open,close,high,low,i,rates_total);
      
      k1[i] = MathMedianSlope(i,inpPeriod,time,prices);
      k0[i] = MathMedianIntercept(i,inpPeriod,time,prices, k1[i]);
      
      distance[i]=(((k0[i]+k1[i]*time[i]-prices[i])/sqrt(1+k1[i]*k1[i]))/3600);
      
      distancecolor[i] = (distance[i]>0) ? 1 : 0;
      fillu[i]  = 0;
      filld[i]  = distance[i];
   }                  
   return(rates_total);
}


//------------------------------------------------------------------
//
//------------------------------------------------------------------
double SlopeBuffer[];
double MathMedianSlope(int i, int _period, const datetime &time[], double &price[])
{
   if (i<_period) return(0);
   int SlopeSize = (_period*(_period-1))/2; if (ArraySize(SlopeBuffer)!=SlopeSize) ArrayResize(SlopeBuffer,SlopeSize); 
    
   for(int l=0,k=0; l<(_period-1); l++)
   for(int j=(l+1); j<(_period  ); j++,k++) SlopeBuffer[k] = (price[i-j]-price[i-l])/(time[i-j]-time[i-l]);
   
   ArraySort(SlopeBuffer);
   double  MedianSlope = (SlopeSize%2==1) ? (SlopeBuffer[SlopeSize/2]) : (0.5*(SlopeBuffer[(SlopeSize-1)/2]+SlopeBuffer[(SlopeSize+1)/2]));
   return( MedianSlope );
}

double InterceptBuffer[];
double MathMedianIntercept(int i, int _period, const datetime &time[], double &price[], double MedianSlope)
{
   if (i<_period) return(0);
   int InterceptSize = (_period); if (ArraySize(InterceptBuffer)!=InterceptSize) ArrayResize(InterceptBuffer,InterceptSize); 
    
   for(int j=1; j<(_period); j++)
      InterceptBuffer[j] = price[j]-MedianSlope*time[j];
   
   
   ArraySort(InterceptBuffer);
                              
   double  MedianIntercept = (InterceptSize%2==1) ? (InterceptBuffer[InterceptSize/2]) : (0.5*(InterceptBuffer[(InterceptSize-1)/2]+InterceptBuffer[(InterceptSize+1)/2]));
   return( MedianIntercept );
}

//
//---
//    
double getPrice(ENUM_APPLIED_PRICE tprice,const double &open[],const double &close[],const double &high[],const double &low[],int i,int _bars)
{
   switch(tprice)
     {
      case PRICE_CLOSE:     return(close[i]);
      case PRICE_OPEN:      return(open[i]);
      case PRICE_HIGH:      return(high[i]);
      case PRICE_LOW:       return(low[i]);
      case PRICE_MEDIAN:    return((high[i]+low[i])/2.0);
      case PRICE_TYPICAL:   return((high[i]+low[i]+close[i])/3.0);
      case PRICE_WEIGHTED:  return((high[i]+low[i]+close[i]+close[i])/4.0);
     }
   return(0);
  }




Arthur Albano
341
Arthur Albano  

Hi people!


I created also an oscillator that calculates the following, which is the quadratic coefficient (second derivative), calculating the acceleration:


//------------------------------------------------------------------
// k = ( - 2 y1 + y2 + y3) / (2 x3 ** 2 )
//------------------------------------------------------------------
double AccelBuffer[];
double MedianAccelerator(int i,int _period,const datetime &time[],double &price[])
  {
   if(i<(_period)) return(0);
   int buffersize=((_period*(_period-1)*(_period-2))/(2*3)); if(ArraySize(AccelBuffer)!=buffersize) ArrayResize(AccelBuffer,buffersize);

   for(int l=0,k=0; l<(_period-2); l++)
      for(int j=(l+1); j<(_period-1); j++)
         for(int m=(j+1); m<(_period); m++,k++)
            AccelBuffer[k] = (10e10)*(-2*prices[i-m] + prices[i-j] + prices[i-l])/(2 * time[i-j]);

   ArraySort(AccelBuffer);

   double  MdAccelerator=(buffersize%2==1) ?(AccelBuffer[buffersize/2]) :(0.5*(AccelBuffer[(buffersize-1)/2]+AccelBuffer[(buffersize+1)/2]));
   return( MdAccelerator);
  }



Enjoy! :)

Arthur Albano
341
Arthur Albano  
Arthur Albano:

Hi people!


I created also an oscillator that calculates the following, which is the quadratic coefficient (second derivative), calculating the acceleration:




Enjoy! :)

http://www.wolframalpha.com/input/?i=y_1%3Dk_0%2Bk_1+x_1+%2B+k_2+x_1%5E2,+y_2%3Dk_0%2Bk_1+x_2+%2B+k_2+x_2%5E2,+y_3%3Dk_0%2Bk_1+x_3+%2B+k_2+x_3%5E2+solve+for+k_2
Wolfram|Alpha: Making the world’s knowledge computable
Wolfram|Alpha: Making the world’s knowledge computable
  • www.wolframalpha.com
Wolfram|Alpha brings expert-level knowledge and capabilities to the broadest possible range of people—spanning all professions and education levels.
Petr Nosek
1529
Petr Nosek  
Arthur Albano:
 

But what is really anoyng me is to get time wright.

So, I have a problem getting distance for any period (step 5). The distance should not change too much, but it seems to change more than it should by changing time period. I have tried using PeriodSeconds(), PERIOD_CURRENT and _Period, but it didn't work as expected.

There goes the code:

Why do you use time in your calculation? As I said you'll get in trouble with using time. The time axis isn't linear one. There aren't weekends, closed times for some symbols... on the time axis. Better way is using just candle numbers IMHO. As you can see in my code.

Arthur Albano
341
Arthur Albano  
Petr Nosek:

Why do you use time in your calculation? As I said you'll get in trouble with using time. The time axis isn't linear one. There aren't weekends, closed times for some symbols... on the time axis. Better way is using just candle numbers IMHO. As you can see in my code.

The simplification of time could lead to a different t/p. I haven't traded with both. I still do not know how to build a bot to test both versions. But most important of all, your indicator displays when prices come out from the buy zone (slope -) and enters sell zone (slope +). When this happens, one should buy and sell with a calculated t/p. At the other hand, when the slope comes from the sell zone (-) and crosses to the other side (+), one should buy and put a t/p. Both could be, theoretically, calculated with one indicator. Like a sinusoid.

I work many years with signal processing using OSI Soft PI products (ProcessBook, alerts, etc) and others. There is a server side median function. I miss having a moving (mathematical) median. That's why I have difficulties forgetting time. And median is far more robust. So even with many errors in data, they are filtered out too.

And I do not know how to assemble a bot. I have started with Metatrader just a month ago.

To add comments, please log in or register