Discussion on array optimization techniques and performance comparisons between different methods.

[Deleted] 2018.11.13 13:41 #71

I don't know what mt4 is, it won't even run on win10, I think that's a good thing

Nikolai Semko 2018.11.13 13:54 #72

Maxim Dmitrievsky:
I don't know what mt4 is, it won't even run on win10, I think that's a good thing

Well said:)

Konstantin Nikitin 2018.11.13 13:55 #73

I tweaked the array.

void OnStart()
{
     long array[], arr[] = {1, 2, 3, 6, 9, 5, 6, 3, 25, , , 6, 8, NULL, 7, NULL, 4, 45};
     ArrayCopy(array, arr); 
//--
  #ifdef __MQL5__
     Print( arrayFilter(array, array[2]) );
     ArrayPrint( array );
  #else
     string s = arrayFilter(array, array[2]) + " {";
     
     for(int i=0; i<ArraySize(array); i++)
     {
          s += array[i];
          if(i<ArraySize(array)-1)
               s += ", ";
     }
     
     Print( s+" }" );
  #endif
}
//---------------------------------------------------------------
template<typename T>
int arrayFilter(T &data[], const T value)
{
     int d = ArraySize(data), j = 0, y = 0;
     
     for(int i=0; i<d; i++, y++)
     {
          if(j>0)
               data[y] = data[y + j];
          
          if(data[i] == value || data[i] == NULL)
          {
               y--;
               j++;
          }
     }
  
     if(d > y)
          ArrayResize(data, y);
     
     return y;
}

I had to use ArrayCopy , becauseMQL5 swore that the array was static.

Sergey Pavlov 2018.11.13 15:37 #74

Nikolai Semko:

If it's such a speed contest, I'll offer my own variant...

Your variant is indeed the fastest, but it contains a bug: if all elements of an array are equal to a filter, your function will exit the array.

I will offer my own variant, it is a little bit slower than yours:

int ArrayModify(int &a[],const int v)
  {
   int size=ArraySize(a)-1;
   int j=0;
   int total=size;
   for(int i=0;i<size;i++)
     {
      while(a[i+j]==v && j<total)
        {
         j++;
         size--;
        }
      a[i]=a[i+j];
     }
   size=ArrayResize(a,size);
   return(size);
  }

2018.11.13 17:16:38.618 massiv v1 (EURUSD,M1) test my=1512090
2018.11.13 17:16:40.083 massiv v1 (EURUSD,M1) test alien=1464941

Experts: Spreader 2 Any questions from newcomers Function - Method for

Vladimir Pastushak 2018.11.13 15:38 #75

Vasiliy Sokolov:

You have been asking these kinds of questions for several years now. Have you learned much? Sorry, but it is obvious that you are still at the level of bytes and elementary arrays.

The question itself is formulated incorrectly. The task is not to remove duplicate values (task of GCE level) but in something much bigger, you must update the list of valid items. If so, the question should sound completely different. You confuse and mislead the participants and, first of all, yourself: impose to the participants the wrong solution in principle and ask to make it effective.

I don't do programming for the sake of programming, I have no goal to become a mega-programmer and to be clever on forums.

What don't you understand in the question: Clear an array of defined elements?

First sacred cow: "If Useful features from KimIV Who trades on levels

Nikolai Semko 2018.11.13 16:10 #76

Sergey Pavlov:

Your variant is really the fastest, but it contains a bug: if all elements of an array are equal to a filter, your function will exit the array.

I'll suggest my variant, it's a bit slower than yours:

2018.11.13 17:16:38.618 massiv v1 (EURUSD,M1) test my=1512090
2018.11.13 17:16:40.083 massiv v1 (EURUSD,M1) test alien=1464941

Yes, thanks. Corrected.

int ArrayDeleteVal(int &a[],const int val) // вариант Semko
  {
   int size=ArraySize(a);
   int i=0,start,s,count;
   while(i<size && a[i]!=val) i++; // ищем первый элемент массива со значением val
   if(i==size) return size;
   start=i; i++;  
   while(i<size && a[i]==val) i++; // ищем элемент массива со значением, не равным val
   if(i==size) {ArrayResize(a,0); return 0;}
   s=i; i++;
   while(true)
     {
      while(i<size && a[i]!=val) i++; // ищем элемент массива со значением val
      count=i-s;
      if(count>6) { ArrayCopy(a,a,start,s,count); start+=count;} // если нужно скопировать более 6 элементов, то имеет смысл воспользоваться ArrayCopy
      else for(; s<i; start++,s++) a[start]=a[s];                // иначе простой цикл
      if(i==size) break;
      i++;
      while(i<size && a[i]==val) i++; // ищем элемент массива со значением, не равным val
      if(i<size) s=i; else break;
      i++;
     }
   if(start<size) ArrayResize(a,start); else start=size;
   return(start);
  }

Only you have an error somewhere, too, because the checksum does not coincide because one element is missing somewhere. Didn't work out where.

2018.11.13 10:07:27.757 ArrayDeleteValue (EURUSD,D1)    вариант Pastushak: Контрольная сумма = 495782577; элементов - 999027; время выполнения = 156757 микросекунд
2018.11.13 10:07:27.761 ArrayDeleteValue (EURUSD,D1)    вариант Korotky:   Контрольная сумма = 495782577; элементов - 999027; время выполнения = 2338 микросекунд
2018.11.13 10:07:27.764 ArrayDeleteValue (EURUSD,D1)    вариант Fedoseev:  Контрольная сумма = 495782577; элементов - 999027; время выполнения = 1839 микросекунд
2018.11.13 10:07:27.766 ArrayDeleteValue (EURUSD,D1)    вариант Semko:     Контрольная сумма = 495782577; элементов - 999027; время выполнения = 782 микросекунд
2018.11.13 10:07:27.770 ArrayDeleteValue (EURUSD,D1)    вариант Pavlov:    Контрольная сумма = 495781718; элементов - 999026; время выполнения = 2886 микросекунд
2018.11.13 10:07:27.773 ArrayDeleteValue (EURUSD,D1)    вариант Nikitin:   Контрольная сумма = 495782577; элементов - 999027; время выполнения = 2355 микросекунд

Files:

ArrayDeleteValue.mq5 13 kb

Konstantin Nikitin 2018.11.13 16:24 #77

Tweaked it by removing unnecessary passages

void OnStart()
{
     long array[], arr[] = {1, 2, 3, 6, 9, 5, 6, 3, 25, , , 6, 8, NULL, 7, NULL, 4, 45};
     ArrayCopy(array, arr);
//--
  #ifdef __MQL5__
     Print( arrayFilter(array, (long)3) );
     ArrayPrint( array );
  #else
     string s = arrayFilter(array, (long)3) + " {";
     
     for(int i=0; i<ArraySize(array); i++)
     {
          s += array[i];
          if(i<ArraySize(array)-1)
               s += ", ";
     }
     
     Print( s+" }" );
  #endif
}
//---------------------------------------------------------------
template<typename T>
int arrayFilter(T &data[], const T value=NULL)
{
     int d = ArraySize(data), y = 0;
     
     for(int i=0, j=0; i<d; i++, y++)
     {
          bool res = false;
          while(data[i] == value || data[i] == NULL)
          {
               res = true;
               j++;
            //---
               if(i+1==d)
                    break;
               if(data[i+1] == value || data[i+1] == NULL)
                    i++;
               else
                    break;
          }
          
          if(j>0)
          {
               if(d==y+j)
                    break;
               data[y] = data[y + j];
          }
          if(res)
               y--;
     }
  
     if(d > y)
          ArrayResize(data, y);
     
     return y;
}

Vladimir 2018.11.13 18:07 #78

Nikolai Semko:
In both cases each element is dragged at most once.

Yes, sorry, one time indeed. I was hoping someone would be interested in the DBMS approach and check it out, didn't wait. Had to do it myself.

int ArrayDelV (int &a[],const int val) { // вариант Vladimir
bool Flags[]; // массив пометок на удаление
int N, NewN, i, j;
N=ArraySize(a);
ArrayResize(Flags,N);
//if (ArrayInitialize(Flags,false)!=N) return(-1);
NewN=N; // Сколько останется
for (i=0;i<N;i++) {if (a[i]==val) {Flags[i]=true; NewN--;}}
j=0;
for (i=0;i<N;i++) {if (Flags[i]) {a[j]=a[i]; j++;}}
ArrayResize(a,NewN);
return(NewN);
}

Inserted ArrayDeleteValue.mq5 into your checker, it is twice as bad as yours. I thought about the reason and fixed two lines in it so that one third of the items would be removed instead of 0.1%.

   for(int i=0; i<1000000;i++) arr[i]=rand()%3;// 1000; //генерируем исходный массив случайными значениями от 0 до 1000
   int Value=rand()%3; // 1000; // значение, которое нужно удалить из массива

That is how it turned out:

2018.11.13 19:45:22.148 Del (GBPUSD.m,H1) variant Pastushak: Checksum = 333586; elements - 667421; execution time = 108521 microseconds
2018.11.13 19:45:22.148 Del (GBPUSD.m,H1) variant Korotky: Checksum = 333586; elements - 667421; execution time = 5525 microseconds
2018.11.13 19:45:22.148 Del (GBPUSD.m,H1) variant Fedoseev: Checksum = 333586; elements - 667421; execution time = 4879 microseconds
2018.11.13 19:45:22.164 Del (GBPUSD.m,H1) variant Semko: Checksum = 333586; elements - 667421; execution time = 14479 microseconds
2018.11.13 19:45:22.179 Del (GBPUSD.m,H1) variant Pavlov: Checksum = 998744; elements - 667421; execution time = 0 microseconds
2018.11.13 19:45:22.179 Del (GBPUSD.m,H1) variant Nikitin: Checksum = 333586; elements - 667421; execution time = 5759 microseconds
2018.11.13 19:45:22.179 Del (GBPUSD.m,H1) variant Vladimir: Checksum = 333586; elements - 667421; execution time = 1542 microseconds

Pavlov's variant had an error, I had to comment it out.

Conclusion: calculating addresses in an array with an arbitrary distance between their numbers is still worse than processing elements in a row, at a given step, much less step 1, the compiler can optimize it.

P.S. Borland's Pascal and Delphi compilers make it so that at runtime the loop variable does not matter (in memory), it is put somewhere in the CPU registers.

Learning and writing together Is it possible to Tendential planimetry method

Nikolai Semko 2018.11.13 18:43 #79

Vladimir:

Yes, sorry, one time indeed. I was hoping someone would be interested in the DBMS approach and check it out, didn't wait. Had to do it myself.

Inserted ArrayDeleteValue.mq5 into your checker, it is twice as bad as yours. I thought about the reason and fixed two lines in it so that one third of the items would be deleted instead of 0.1%.

That is how it turned out:

2018.11.13 19:45:22.148 Del (GBPUSD.m,H1) variant Pastushak: Checksum = 333586; elements - 667421; execution time = 108521 microseconds
2018.11.13 19:45:22.148 Del (GBPUSD.m,H1) variant Korotky: Checksum = 333586; elements - 667421; execution time = 5525 microseconds
2018.11.13 19:45:22.148 Del (GBPUSD.m,H1) variant Fedoseev: Checksum = 333586; elements - 667421; execution time = 4879 microseconds
2018.11.13 19:45:22.164 Del (GBPUSD.m,H1) variant Semko: Checksum = 333586; elements - 667421; execution time = 14479 microseconds
2018.11.13 19:45:22.179 Del (GBPUSD.m,H1) variant Pavlov: Checksum = 998744; elements - 667421; execution time = 0 microseconds
2018.11.13 19:45:22.179 Del (GBPUSD.m,H1) variant Nikitin: Checksum = 333586; elements - 667421; execution time = 5759 microseconds
2018.11.13 19:45:22.179 Del (GBPUSD.m,H1) variant Vladimir: Checksum = 333586; elements - 667421; execution time = 1542 microseconds

Pavlov's variant had an error, I had to comment it out.

Conclusion: calculating addresses in an array with an arbitrary distance between their numbers is still worse than processing items in a row at a given step, all the more so at step 1, the compiler can optimize it.

P.S. Borland's Pascal and Delphi compilers make it so that at runtime the loop variable does not matter (in memory), it is put somewhere in the CPU registers.

Pavlov's version has been corrected.
Your values are strange. Maybe you ran the script after profiling or debugger without recompiling the code?
That's how it works for me:

2018.11.13 12:35:38.633 ArrayDeleteValue (EURUSD,D1)    вариант Pastushak: Контрольная сумма = 496494849; элементов - 999011; время выполнения = 131964 микросекунд
2018.11.13 12:35:38.636 ArrayDeleteValue (EURUSD,D1)    вариант Korotky:   Контрольная сумма = 496494849; элементов - 999011; время выполнения = 2310 микросекунд
2018.11.13 12:35:38.639 ArrayDeleteValue (EURUSD,D1)    вариант Fedoseev:  Контрольная сумма = 496494849; элементов - 999011; время выполнения = 1834 микросекунд
2018.11.13 12:35:38.641 ArrayDeleteValue (EURUSD,D1)    вариант Semko:     Контрольная сумма = 496494849; элементов - 999011; время выполнения = 773 микросекунд
2018.11.13 12:35:38.645 ArrayDeleteValue (EURUSD,D1)    вариант Pavlov:    Контрольная сумма = 496494849; элементов - 999011; время выполнения = 2815 микросекунд
2018.11.13 12:35:38.648 ArrayDeleteValue (EURUSD,D1)    вариант Nikitin:   Контрольная сумма = 496494849; элементов - 999011; время выполнения = 2475 микросекунд
2018.11.13 12:35:38.653 ArrayDeleteValue (EURUSD,D1)    вариант Vladimir:  Контрольная сумма = 496656342; элементов - 999011; время выполнения = 3608 микросекунд

And in your version it generates an incorrect checksum. And creating an additional array is of no benefit at all, on the contrary, it slows down the process and eats up additional resources.

Files:

ArrayDeleteValue.mq5 15 kb

New metatrader 4 compatible How to know daily Questions from Beginners MQL5

Nikolai Semko 2018.11.13 18:54 #80

In essence, all algorithms are the same. Everyone modifies the array element by element, and everyone strives for the Fedoseev variant, because it does not have anything unnecessary.
Only in my case it is done in blocks using ArrayCopy, so there is a speed advantage.

Discussion of article "The Important Update for MetaTrader ArrayCopy with arrays of

Clearing an array of defined element(s) - page 8