Discussion of article "What is a trend and is the market structure based on trend or flat?" - page 10

New comment
 

I see what your problem was. Initially you say that the process can have a maximum deviation of 40. And you calculate on 40. But the process has the possibility to go both in + and in - resulting in 81 variants (with an exact 0).

And you plot 40 points clustered in steps of 2 on the X axis.

That's why you have such a graph.

I transferred the data from Excel to MQL.

And using the functionality of the standard library I have selected sigma parameters to repeat your graph from Excel also grouping forcedly with step 2.

It turns out that for your process with combinatorics the process with a normal distribution of MO 0 and sigma 6.45 is suitable.

//+------------------------------------------------------------------+
//|NormalSimple.mq5 |
//|https://www.mql5.com |
//+------------------------------------------------------------------+
#include <Graphics\Graphic.mqh>
#include <Math\Stat\Normal.mqh>
#include <Math\Stat\Math.mqh>
#property script_show_inputs
//--- input parameters
input double mean_value=0;  // mathematical expectation (mean)
input double std_dev=6.45;     // standard deviation (standard deviation)
//+------------------------------------------------------------------+
//| Script programme start function|
//+------------------------------------------------------------------+
void OnStart()
  {
   double arrayTeoryX[41]= {-40,-38,-36,-34,-32,-30,-28,-26,-24,-22,-20,-18,-16,-14,-12,-10,-8,-6,-4,-2,0,
                            2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40
                           };
   double arrayTeoryY[41]= {0.00000009,3.63798 E-06,7.09406 E-05,0.000898581,0.008311872,0.059845479,0.349098627,
                            1.695621904,6.994440355,24.86912126,77.09427591,210.2571161,508.121364,1094.415245,
                            2110.657973,3658.473821,5716.365345,8070.162839,10311.87474,11940.06549,12537.06876,
                            11940.06549,10311.87474,8070.162839,5716.365345,3658.473821,2110.657973,1094.415245,
                            508.121364,210.2571161,77.09427591,24.86912126,6.994440355,1.695621904,0.349098627,
                            0.059845479,0.008311872,0.000898581,7.09406 E-05,3.63798 E-06,9.09495 E-08
                           };
//--- disable price chart display
   ChartSetInteger(0,CHART_SHOW,false);
//--- initialise the random number generator
   MathSrand(GetTickCount());
//--- generate a sample of the random variable
   long chart=0;
   string name="GraphicNormal";
   int n=100000;       // number of values in the sample
   int ncells=41;       // number of intervals in the histogram
   double x[];          // histogram interval centres
   double y[];          // number of values from the sample that fall into the interval
   double data[];       // sampling of random values
   double max,min;      // maximum and minimum values in the sample
//--- get a sample from the normal distribution
   MathRandomNormal(mean_value,std_dev,n,data);
//--- calculate data for histogram construction
   CalculateHistogramArray(data,x,y,max,min,ncells);
   /*
//--- get sequence boundaries and step for theoretical curve construction
 double step;
 GetMaxMinStepValues(max,min,step);
 step=MathMin(step,(max-min)/ncells);
//--- get theoretically calculated data on the interval [min,max]
 double x2[];
 double y2[];
 MathSequence(min,max,step,x2);
 MathProbabilityDensityNormal(x2,mean_value,std_dev,false,y2);
//--- scaling
 double theor_max=y2[ArrayMaximum(y2)];
 double sample_max=y[ArrayMaximum(y)];
 double k=sample_max/theor_max;
 */
   /*
 for(int i=0; i<ncells; i++)
 y[i]/=k;
 */
//--- display graphs
   CGraphic graphic;
   if(ObjectFind(chart,name)<0)
      graphic.Create(chart,name,0,0,0,1000,680);
   else
      graphic.Attach(chart,name);
   graphic.BackgroundMain(StringFormat("Normal distribution mu=%G sigma=%G",mean_value,std_dev));
   graphic.BackgroundMainSize(16);
//--- plot all curves
   graphic.CurveAdd(x,y,CURVE_HISTOGRAM,"MQL step 2").HistogramWidth(6);
   graphic.CurveAdd(arrayTeoryX,arrayTeoryY,CURVE_LINES,"Excel");
//--- and now let's plot the theoretical distribution density curve
//graphic.CurveAdd(x2,y2,CURVE_LINES, "Theory");
//--- plot all curves
   graphic.CurvePlotAll();
   graphic.Update();
   double summ=0;
   for(int i=0; i<ArraySize(y); i++)
     {
      summ+=y[i];
     }
   Print("Total amount by Y " + summ);
  }
//+------------------------------------------------------------------+
//| Calculate frequencies for data set|
//+------------------------------------------------------------------+
bool CalculateHistogramArray(const double &data[],double &intervals[],double &frequency[],
                             double &maxv,double &minv,const int cells=10)
  {
   if(cells<=1)
      return (false);
   int size=ArraySize(data);
   if(size<cells*10)
      return (false);
   minv=data[ArrayMinimum(data)];
   maxv=data[ArrayMaximum(data)];
   double range=maxv-minv;
   double width=range/cells;
   width=2.0;
   minv=-41;
   if(width==0)
      return false;
   ArrayResize(intervals,cells);
   ArrayResize(frequency,cells);
//--- set the centres of the intervals
   for(int i=0; i<cells; i++)
     {
      intervals[i]=minv+(i+0.5)*width;
      frequency[i]=0;
     }
//--- fill in the interval frequencies
   for(int i=0; i<size; i++)
     {
      int ind=int((data[i]-minv)/width);
      if(ind>=cells)
         ind=cells-1;
      frequency[ind]++;
     }
   return (true);
  }
  /*
//+------------------------------------------------------------------+
//| Calculates values for sequence generation|
//+------------------------------------------------------------------+
void GetMaxMinStepValues(double &maxv,double &minv,double &stepv)
 {
//--- calculate the absolute range of the sequence to get the normalisation accuracy
 double range=MathAbs(maxv-minv);
 int degree=(int)MathRound(MathLog10(range));
//--- normalise max. and min. and min. values with the specified accuracy
 maxv=NormaliseDouble(maxv,degree);
 minv=NormalizeDouble(minv,degree);
//--- we also set the step of sequence generation from the given precision
 stepv=NormalizeDouble(MathPow(10,-degree),degree);
 if((maxv-minv)/stepv<10)
 stepv/=10.;
 }
//+------------------------------------------------------------------+
*/
 
Alexey Klenov:

I see what your problem was. Initially you say that the process can have a maximum deviation of 40. And you calculate on 40. But the process has the ability to go both + and -, so you end up with 81 variations (with an exact 0).

And you plot 40 points clustered in increments of 2 on the X axis.

That's why you have such a graph.

I transferred the data from Excel to MQL.

And using the functionality of the standard library I selected sigma parameters to repeat your graph from Excel, also grouping forcedly with step 2.

It turns out that for your process with combinatorics the process with a normal distribution of MO 0 and sigma 6.45 is suitable.

Oh, that's interesting. Yes, there is a range from -40...0...40 in steps of 2, you wrote everything correctly. Maybe I didn't chew this point up in the article. Thanks for the code.
 

Afternoon

The blue histogram is a repeat of your experiment only on 365 days of GBPUSD tick data

Renko at 0.0002 points and slicing by 40 renko bars.

Almost completely repeats your theoretical curve from excel.

So your figure 7 to the article does not work out.

 

Here's the code

I'm looking for errors and haven't found any yet.

//+------------------------------------------------------------------+
//|ProjectName |
//|Copyright 2020, CompanyName |
//|http://www.companyname.net |
//+------------------------------------------------------------------+
#include <Graphics\Graphic.mqh>
#include <Math\Stat\Normal.mqh>
#include <Math\Stat\Math.mqh>
#property script_show_inputs
//--- input parameters
//input double mean_value=0; // mathematical expectation (mean)
//input double std_dev=1; // standard deviation (standard deviation)
input double stepRenko=0.0002;   // The pitch of the renko bar
//+------------------------------------------------------------------+
//| Script programme start function|
//+------------------------------------------------------------------+
void OnStart()
  {
//--- disable price chart display
   ChartSetInteger(0,CHART_SHOW,false);
//--- initialise the random number generator
   MathSrand(GetTickCount());
//--- generate a sample of the random variable
   long chart=0;
   string name="GraphicNormal";
   int ncells=41;       // number of intervals in the histogram
   double x[];          // histogram interval centres
   double y[];          // number of values from the sample that fall into the interval
   double data[];       // sampling of random values
   double max,min;      // maximum and minimum values in the sample

   MqlTick realTick[];
   ulong start=iTime(_Symbol,PERIOD_D1,365)*1000;         // in ms
   ulong finish=iTime(_Symbol,PERIOD_D1,1)*1000;         // in ms

   int countCopy=CopyTicksRange(_Symbol,realTick,COPY_TICKS_INFO,start,finish);
   Print("countCopy="+IntegerToString(countCopy));
   Print("0 tick time ="+TimeToString(realTick[0].time));
   Print("end tick time ="+TimeToString(realTick[countCopy-1].time));
   
   int index=0;
   double actualStep[];       // array for renko bars
   ArrayResize(actualStep,countCopy);
   ArrayInitialize(actualStep,0);
   double priceForCheck= realTick[0].bid;
   actualStep[0]=0;
   int cnt=0;

   for(int i=0; i<countCopy; i++)
     {
      if(realTick[i].bid>priceForCheck+stepRenko)
        {
         int add=(int)((realTick[i].bid-priceForCheck)/stepRenko);
         actualStep[index]+=add;
         priceForCheck+=stepRenko*(double)add;
         cnt+=add;
         if(cnt>39)
           {
            index++;
            cnt=0;
           }
         continue;
        }
      if(realTick[i].bid<priceForCheck-stepRenko)
        {
         int add=(int)((priceForCheck-realTick[i].bid)/stepRenko);
         actualStep[index]-=add;
         priceForCheck-=stepRenko*(double)add;
         cnt+=add;
         if(cnt>39)
           {
            index++;
            cnt=0;
           }
        }
     }
   ArrayResize(actualStep,index+1);
   Print("Count index="+IntegerToString(index));
   CalculateHistogramArrayItsMy(actualStep,x,y,max,min,ncells);

// Data from Excel
   double arrayTeoryX[41]= {-40,-38,-36,-34,-32,-30,-28,-26,-24,-22,-20,-18,-16,-14,-12,-10,-8,-6,-4,-2,0,
                            2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40
                           };
   double arrayTeoryY[41]= {0.00000009,3.63798 E-06,7.09406 E-05,0.000898581,0.008311872,0.059845479,0.349098627,
                            1.695621904,6.994440355,24.86912126,77.09427591,210.2571161,508.121364,1094.415245,
                            2110.657973,3658.473821,5716.365345,8070.162839,10311.87474,11940.06549,12537.06876,
                            11940.06549,10311.87474,8070.162839,5716.365345,3658.473821,2110.657973,1094.415245,
                            508.121364,210.2571161,77.09427591,24.86912126,6.994440355,1.695621904,0.349098627,
                            0.059845479,0.008311872,0.000898581,7.09406 E-05,3.63798 E-06,9.09495 E-08
                           };
   // Scaling the theoretical curve to the real curve 
   double div=100000/index;
   for(int i=0; i<ArraySize(arrayTeoryY); i++)
     {
      arrayTeoryY[i]/=div;
     }

   CGraphic graphic;
   if(ObjectFind(chart,name)<0)
      graphic.Create(chart,name,0,0,0,1080,580);
   else
      graphic.Attach(chart,name);
//graphic.BackgroundMain(StringFormat("Normal distribution mu=%G sigma=%G stepRenko=%G",mean_value,std_dev,stepRenko));
//graphic.BackgroundMain(StringFormat("Normal distribution mu=%G sigma=%G",mean_value,std_dev));
   graphic.BackgroundMainSize(16);
//--- plot all curves
   graphic.CurveAdd(x,y,CURVE_HISTOGRAM,"Sample").HistogramWidth(6);
   graphic.CurveAdd(arrayTeoryX,arrayTeoryY,CURVE_LINES,"Excel");


//--- plot all curves
   graphic.CurvePlotAll();
   graphic.Update();
  }
//+------------------------------------------------------------------+
//||
//+------------------------------------------------------------------+
bool CalculateHistogramArrayItsMy(const double &data[],double &intervals[],double &frequency[],
                                  double &maxv,double &minv,const int cells=10)
  {
   minv=data[ArrayMinimum(data)];
   maxv=data[ArrayMaximum(data)];
   int range=maxv-minv;
   ArrayResize(intervals,range+1);
   ArrayResize(frequency,range+1);
   for(int i=0; i<range+1; i++)
     {
      intervals[i]=minv+i;
     }
   for(int i=0; i<ArraySize(data); i++)
     {
      int ii=(MathAbs(minv)+data[i]);
      //frequency[ii]+=MathAbs(data[i]);
      frequency[ii]+=1.0;
     }
   return (true);
  }
//+------------------------------------------------------------------+
 
Alexey Klenov:

Good afternoon

The blue histogram is a repeat of your experiment only on 365 days of GBPUSD tick data

Renko at 0.0002 pips and slicing 40 renko bars.

Almost completely repeats your theoretical curve from excel.

So your figure 7 to the article doesn't work out.

I used minute candles to analyse, when you slice them in too small blocks there is some error.
I don't have renko bars, the algorithm is a bit different. In renko bars it takes 2 pips to form a reversal block, but with me it is always 1x.
Please send me a tick history, I will try it next week and show you what I got. On ticks should be slightly trendy in idea, but in fact, ticks are different, it depends on the source. For example, on some demo accounts there are too many ticks, and this leads to a reversal process.
 
Maxim Romanov:
I used minute candlesticks to analyse, when you slice them into too small blocks there is some error.
I don't have renko bars, the algorithm is a bit different. With Renko bars it takes 2 pips to form a reversal block, but with me it is always 1x.
Please send me a tick history, I will try it next week and show you what I got. On ticks should be slightly trendy in idea, but in fact, ticks are different, it depends on the source. For example, on some demo accounts there are too many ticks, and this leads to a reversal process.

The terminal downloads the tick history itself (MT5, Alpari-MT5 server).

Variant. in the terminal Ctrl +U then on the ticks tab you can do everything.

 
The normality of your stock distribution clearly depends on the number of observations: the more observations, the more normal it is. You should compare equal sample sizes.
 
Kristian Kafarov:
The normality of your stock distribution clearly depends on the number of observations: the more observations, the more normal it is. You should compare similar sample sizes.

From the examples given in the article, it may indeed seem so. But in reality, there is no such effect. The more samples, the more accurate the result, but the distribution is not close to normal (although there are certain conditions to be fulfilled in the study). The nature of the distribution depends more on the trading instrument, each one is slightly different.

 
This is an interesting approach to analysing a flat trend. But I consider it not very acceptable in practice. Rather, the material can be attributed to familiarisation with another aspect and no more.
 
Thank you very much for this market insight and explanation
1...34567891011
New comment