ArraySort() error on multidimensional array - page 3

New comment
 
J.P.Satrio:

Greetings,


I'm studying MQL4 and MQL5 for a few years now (I'm not a professional programmer, I'm self educated), but I find hard to understand topics still. Recently, I wanted to store chart object data from a sqlite database to a multi-dimensional array in a project, to minimize disk I/O. As far as I know, I need to declare the array somewhat like this:

But when I try to work with the array, so like to search in it with ArrayBsearch, I need to sort it first:

This way I get an error:

'ArraySort' - constant cannot be modified

At this point, there were no data transaction between the database and the array, the error showed up when I added the array to the code. I tried to comment out each and every data types, same error. I tried ArraySort with a one-dimensional integer type array, to test, all went well. So it is quite sure, the struct declaration, or the struct-based array causes the error.

It is also veird, that I cannot add a string to the struct with that syntax, I get a

'MarkerArray' - structures or classes containing objects are not allowed

built-in: bool ArraySort(void&)

You need to use custom/library sort and binary search functions, as MQL standard library does not handle arrays of structures.

So I included the introsort library to help accomplish the goal. https://www.mql5.com/en/code/41836

Here is a test code that you can use in your project:

struct MarkerData
  {
   ENUM_TIMEFRAMES   timeframe;
   string            objectname;
   datetime          timestamp;
   //ENUM_MarkerOptions flag;
  };

MarkerData MarkerArray[10];

// https://www.mql5.com/en/code/41836
#include "Introsort.mqh"

//+------------------------------------------------------------------+
//| a custom comparison function Less.                               |
//+------------------------------------------------------------------+
bool Less(const MarkerData& left, const MarkerData& right)
  {
// sort on key 'timestamp' in ascending order
   if(left.timestamp < right.timestamp) return(true);
   if(left.timestamp > right.timestamp) return(false);

// if two objects are equal on key 'timestamp',
// sort on key 'objectname' in ascending order
   if(left.objectname < right.objectname) return(true);
   if(left.objectname > right.objectname) return(false);

// if two objects are equal on keys 'timestamp' and 'objectname',
// sort on key 'timeframe' in ascending order
   if(left.timeframe < right.timeframe) return(true);
   if(left.timeframe > right.timeframe) return(false);

//--- all keys are equal
   return(false);
  }

//+------------------------------------------------------------------+
//|                                                                  |
//+------------------------------------------------------------------+
void OnStart()
  {
//--- Generate random data.
   for(int i = 0; i < 10; i++)
     {
      MarkerArray[i].timeframe = (ENUM_TIMEFRAMES)(rand() % 21);
      MarkerArray[i].objectname = "str_" + (string)(rand() % 10);
      MarkerArray[i].timestamp = TimeCurrent() - (rand() << 14);
     }

   Print("Before sort:");
   ArrayPrint(MarkerArray);

//--- MQL cannot sort structures natively, so must use a library sort function.
   //ArraySort(MarkerArray);

//--- Sort the input array in-place using comparison function less.
   Introsort(MarkerArray);

   Print("After sort:");
   ArrayPrint(MarkerArray);

//--- Binary search in the sorted array.
   Print("The element to find using binary search: ", MarkerArray[6].timestamp);

   int index = ArrayBinarySearch(MarkerArray, MarkerArray[6]);
   Print("Index of the found element: ", index);
  }

//+------------------------------------------------------------------+
//| Binary Search                                                    |
//| Searches an entire one-dimensional sorted array for a specific   |
//| element, using the comparison function 'less'.                   |
//| Returns the index of element if it is found, or -1 otherwise.    |
//+------------------------------------------------------------------+
template<typename T>
int ArrayBinarySearch(const T &array[], const T &value)
  {
   int lo = 0;
   int hi = ArraySize(array) - 1;
   while(lo <= hi)
     {
      int i = lo + (hi - lo) / 2;

      if(Less(array[i], value))
        {
         lo = i + 1;
        }
      else if(Less(value, array[i]))
        {
         hi = i - 1;
        }
      else
        {
         return i;
        }
     }
   return -1; // not found
  }

Here is a sample output in the 'experts log' after running the above script:

// sample output:

/*
 Before sort:
     [timeframe] [objectname]         [timestamp]
 [0]           9 "str_6"      2019.12.11 19:46:30
 [1]           6 "str_1"      2006.11.26 03:18:46
 [2]           7 "str_2"      2022.08.04 13:20:22
 [3]          15 "str_1"      2022.04.06 21:35:18
 [4]           2 "str_5"      2018.09.20 16:15:18
 [5]          13 "str_3"      2011.11.19 01:34:14
 [6]          11 "str_9"      2010.07.29 18:25:26
 [7]           8 "str_6"      2012.03.08 14:52:06
 [8]          16 "str_2"      2007.05.27 18:01:58
 [9]          15 "str_4"      2010.10.15 16:55:50

 After sort:
     [timeframe] [objectname]         [timestamp]
 [0]           6 "str_1"      2006.11.26 03:18:46
 [1]          16 "str_2"      2007.05.27 18:01:58
 [2]          11 "str_9"      2010.07.29 18:25:26
 [3]          15 "str_4"      2010.10.15 16:55:50
 [4]          13 "str_3"      2011.11.19 01:34:14
 [5]           8 "str_6"      2012.03.08 14:52:06
 [6]           2 "str_5"      2018.09.20 16:15:18
 [7]           9 "str_6"      2019.12.11 19:46:30
 [8]          15 "str_1"      2022.04.06 21:35:18
 [9]           7 "str_2"      2022.08.04 13:20:22

 The element to find using binary search: 2018.09.20 16:15:18
 Index of the found element: 6
*/
Introsort (Introspective sort)
Introsort (Introspective sort)
  • www.mql5.com
Sort the input array in-place using comparison function less.
 
J.P.Satrio #:

Thank you very much for the detailed explanation. I also checked the 3 links you provided - for the first one, I knew most of it, therefore I did not check back there to realize a structure is considered a variable but as a complex one, but I did check its docs page, to get familiar with the syntax and such, when I was studying its properties. The second one was more or less known for me, but the third one, was quite a deep dive,  its more of a complex topic for me at this time.

Last night I was added the complete funcionality with the database approach. I'm currently adding a feature to add masses os records to test performance drop.

I'll review the suggested pages in the near future, if/when I'll decide to change the code to the array approach, if the change is justified by a significant drop in performance, however, I know that would be the best practice, or a more elegant way to solve this problem.

Thank you very much for your guidance. I'll come back to you soon with the outcome.

We are actually trying to keep you away from databases as replacement for arrays...

For the task, you try to accomplish, it's comparable to using an international cargo ship to bring 3 eggs back to your neighbor house. I mean, this is even for an Australian considered oversized.
 
amrali #:

You need to use custom/library sort and binary search functions, as MQL standard library does not handle arrays of structures.

So I included the introsort library to help accomplish the goal. https://www.mql5.com/en/code/41836

Here is a test code that you can use in your project:

Here is a sample output in the 'experts log' after running the above script:

Thank you for the example. I haven't come around to code something to show...
 
amrali #:

You need to use custom/library sort and binary search functions, as MQL standard library does not handle arrays of structures.

So I included the introsort library to help accomplish the goal. https://www.mql5.com/en/code/41836

Here is a test code that you can use in your project:

Here is a sample output in the 'experts log' after running the above script:

Woah! Thanks for the example!

I saw template BinarySearch somewhere before, but these are way hard for me to understand, what is happening and why, neither the syntax I get. Guess I should declare it as a standard function which I did. In the meantime (when I realized that I might be able to stick to the array approach with your code) I started to reorganize the whole code, so I created multiple MARKERS table (where I store the data outside runtime, from where it is read back during initialization) for each timeframe, so the struct became simpler, it stores now a datetime, and a string (EnumToString - the flag itself) only. So I adjusted the Less function to manage 2 parameters instead of 3. But the problem came when I started to search (the sort function was successful).

I wanted to search with this:

   int index = ArrayBinarySearch(MarkerArray, CueLinePos);

CueLinePos is a datetime type variable to store a cue line. I want to know if the array stores data for the matching datetime (CueLinePos), but I got the following compiler errors:

template parameter ambiguous, could be 'MarkerData' or 'datetime'

   see declaration of function 'ArrayBinarySearch'

CueLinePos' - parameter conversion not allowed

'CueLinePos' - variable of the same type expected

What did I do wrong?
 
Dominik Egert #:
We are actually trying to keep you away from databases as replacement for arrays...

For the task, you try to accomplish, it's comparable to using an international cargo ship to bring 3 eggs back to your neighbor house. I mean, this is even for an Australian considered oversized.

Haha! That analogy was hilarious! I try to implement the array approach since that was the original plan. Now that I have a hopeful snippet to play around, I'll be able to accomplish the original goal. Thanks for your guidance!

 
J.P.Satrio #:
CueLinePos

You could specialize the 'generic' binary search function for your needs like this:

//+------------------------------------------------------------------+
//| Binary Search                                                    |
//| Searches an entire 'MarkerData' sorted array for a specific      |
//| element by its 'timestamp' filed.                                |
//| Returns the index of element if it is found, or -1 otherwise.    |
//+------------------------------------------------------------------+
int MarkerArrayBinarySearch(const MarkerData &array[], const datetime CueLinePos)
  {
   int lo = 0;
   int hi = ArraySize(array) - 1;
   while(lo <= hi)
     {
      int i = lo + (hi - lo) / 2;

      if(array[i].timestamp < CueLinePos)
        {
         lo = i + 1;
        }
      else if(array[i].timestamp > CueLinePos)
        {
         hi = i - 1;
        }
      else
        {
         return i;
        }
     }
   return -1; // not found
  }
 
amrali #:

You could specialize the 'generic' binary search function for your needs like this:

amrali, that is not binary, is it??

Didn't expect such a simple search algorithm from you....



 
amrali #:

You could specialize the 'generic' binary search function for your needs like this:

Greetings,

I've modified the function as suggested from ArrayBinarySearch --> MarkerArrayBinarySearch and rewritten the expected parameters as you recommended (however I had to use

const datetime CueLine

instead of CueLinePos as you told, since that way it was hiding the global variable. I'm giving value to the CueLinePos variable like this:

CueLinePos = iTime(NULL, PERIOD_CURRENT, iBarShift(NULL, PERIOD_CURRENT, CueLinePos, true) + 1);

but when I try to call the Bsearch function like this:

   int index = MarkerArrayBinarySearch(MarkerArray, CueLinePos);
   if(index >= 0)
   {
      Print("Index of the found element: ", index);
      Print("Current CueLinePos value: ", TimeToString(CueLinePos));
      Print(MarkerArray[index]);
   }
   else
   {
      Print("Element not found!");
   }

I get an error:

'[' - objects are passed by reference only.

I tried to add the ampersand before the index like MarkerArray[&index], then I got:

'&' - illegal operation use

'index' - class type expected

'[' - objects are passed by reference only

When I comment out the Print statement, the function always returns 0. A sample output:

/*
2023.08.25 11:10:48.427 [timestamp] [flag]
2023.08.25 11:10:48.427 [0] 2023.08.25 00:40:00 "SBY" 
2023.08.25 11:10:48.427 [1] 2023.08.25 00:39:00 "IDL" 
2023.08.25 11:10:48.427 [2] 2023.08.25 00:38:00 "SSL" 
2023.08.25 11:10:48.427 [3] 2023.08.25 00:37:00 "SBY" 
2023.08.25 11:10:48.427 After sort: 
2023.08.25 11:10:48.427  [timestamp] [flag]
2023.08.25 11:10:48.427 [0] 2023.08.25 00:37:00 "SBY" 
2023.08.25 11:10:48.427 [1] 2023.08.25 00:38:00 "SSL" 
2023.08.25 11:10:48.427 [2] 2023.08.25 00:39:00 "IDL" 
2023.08.25 11:10:48.427 [3] 2023.08.25 00:40:00 "SBY" 
2023.08.25 11:10:48.427 Index of the found element: 0
2023.08.25 11:10:48.427 Current CueLinePos value: 2023.08.25 00:37
*/

Maybe the different format in the datetime causes this issue? It is stored in the array as TIME_DATE|TIME_MINUTES|TIME_SECONDS but when I print its value out it comes as TIME_DATE|TIME_MINUTES. And how to solve the pass by reference error?

Thank you in advance, from this point, I can see the finish line.

 
Dominik Egert #:
amrali, that is not binary, is it??

Didn't expect such a simple search algorithm from you....



I asked ChatGPT to evaluate the the function above and it gave the following answer:

" In summary, the provided function is an efficient binary search algorithm that searches for the specified CueLinePos timestamp value within the sorted MarkerData array. If the timestamp is found, the algorithm returns the index of the value; otherwise, it returns -1 to indicate that the desired value is not present in the array."

 
Dominik Egert #:
amrali, that is not binary, is it??

Didn't expect such a simple search algorithm from you....



Actually this is a binary search. One of many other variations. Variations include getting the nearest match if exact item is not found, next greater or lesser element and etc .
1234
New comment
Reason: