From Basic to Intermediate: Structs (II)

In the previous article, From Basic to Intermediate: Structs (I), we began discussing a topic that, in my view, is very important to understand as clearly as possible. That is because it allows us to do many things in a much simpler way. However, the true importance of understanding every detail lies in the fact that structures sit halfway between what would be object-oriented programming and conventional programming.

Since this topic is just beginning, we still have a lot to cover before we can truly say: Yes, I already know how to work with structures.

Using Structures in Functions and Procedures

One of the things that leaves many beginners feeling lost when dealing with structures is the question of whether or not values should be passed using structures. Indeed, this is quite an interesting question that, at times, ends up causing more confusion than anything else. Contrary to what some may think, the reason is precisely because we can pass variables by reference, whether to a function or to a procedure. And when this is done, we need to be careful when working with structures in such scenarios.

As someone from the old school, I lived through a time when the C language did not allow data transfer via structures, not directly at least. Today this is possible, but there was a time when we had to use other mechanisms to perform such transfers. In that scenario, the possibilities for error grew as the structure accumulated more variables. But that is the past. Today we have safer mechanisms to perform the same type of transfer. Still, nothing stops you from using older implementation techniques, where the processing speed not the security is the main focus.

Nevertheless, despite it being possible to do such things, I WILL NOT show how to implement them, as the difficulty and risk of errors are very high. Therefore, let's learn how to do things the correct way. First, you must understand that a structure does not always need to be passed between different routines. In other words, a function or procedure does not necessarily need to know that it is dealing with a data structure or working with discrete values.

When we use the mechanisms already shown and explained in other articles, we can make the code far more flexible and much more usable. Because once we begin using structures, we may end up limiting certain activities of a given routine.

To understand this, let's look at a simple piece of code shown below.

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. void OnStart(void)
05. {
06.     MqlDateTime dt;
07. 
08.     TimeToStruct(TimeLocal(), dt);
09. 
10.     Print(DateToString(dt.day, dt.mon, dt.year));
11.     Print(DateToString(3, 15, 2024));
12. }
13. //+------------------------------------------------------------------+
14. string DateToString(int day, int month, int year)
15. {
16.     if ((day < 1) || (day > 31) || (month < 1) || (month > 12))
17.         return "ERROR...";
18. 
19.     return StringFormat("%02d/%02d/%d", day, month, year);
20. }
21. //+------------------------------------------------------------------+

Code 01

When this Code 01 is executed, it will produce the result shown below:

Figure 01

Notice that in Code 01, we are hardly using anything complex or difficult to understand. But the result you will see when you run this code will certainly be different from what is shown in Figure 01. The reason for this is simple: the use of the function TimeLocal. This function from the standard MQL5 library captures the current clock value and returns it in datetime format. This format, which is an eight-byte type, contains information about the current day and time in the location where MetaTrader 5 is running. Of course, this time is counted in seconds, but this is irrelevant for our purposes. However, this same value returned by TimeLocal is used by the function TimeToStruct to fill the MqlDateTime structure. And this is where we want to arrive. The MqlDateTime structure is declared as shown below:

struct MqlDateTime 
  { 
   int year;
   int mon;
   int day;
   int hour;
   int min;
   int sec;
   int day_of_week;
   int day_of_year;
  };

MqlDateTime Declaration

Now pay attention to what I'm about to explain, because it is important for understanding what we are doing in Code 01. The function TimeToStruct decomposes the 64-bit value we know as the datetime type into values, assigning each variable in the structure declaration above an appropriate value. There are other ways to do this, but that is not the point here. The point is that, once the values are assigned, we can use them as shown on line 10 of Code 01. Notice, however, that this was only possible because, in the function on line 14, we are using primitive-type arguments to receive the data. This allows us to discard unnecessary values and use only what we actually need.

However, this same approach also allows the use seen on line 11. But due to the check on line 16, the routine on line 14 ends up returning an error, because we CANNOT have a year with fifteen months.

This is the kind of situation I wanted to highlight. The same approach shown here in Code 01 could be implemented in various different ways, such as what can be seen in Code 02 below.

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. void OnStart(void)
05. {
06.     MqlDateTime dt;
07. 
08.     TimeToStruct(TimeLocal(), dt);
09. 
10.     Print(DateToString(dt));
11.     dt.day  = 3;
12.     dt.mon  = 15;
13.     dt.year = 2024;
14.     Print(DateToString(dt));
15. }
16. //+------------------------------------------------------------------+
17. string DateToString(MqlDateTime &arg)
18. {
19.     if ((arg.day < 1) || (arg.day > 31) || (arg.mon < 1) || (arg.mon > 12))
20.         return "ERROR...";
21. 
22.     return StringFormat("%02d/%02d/%d", arg.day, arg.mon, arg.year);
23. }
24. //+------------------------------------------------------------------+

Code 02

In this case, we would have the same result seen in Image 01. However, it would be much easier to understand the reason for the error, because simply looking at line 12 in Code 02, we can clearly see what is wrong. But note that on line 17, it was necessary to pass the structure by reference. This can be a problem, but it is also easy to work around, since we only need to ensure that the received argument is treated as a constant. How to do this has already been explained in other articles. Nonetheless, the fact remains that the function now has far more privileges than before, as it can see more data than necessary for its operation. And for this reason, there is no universally correct answer for how we should use structures in functions or procedures, since you may be passing far more information than necessary.

This concept of the minimal knowledge a function or procedure should have about other data is much more often mentioned when discussing classes and objects. That is because in object-oriented programming this principle is far more present than you may imagine. But that is a topic for the future. For now, you must understand that the less a routine (whether a function or a procedure) needs to know, the better. This reduces the amount of information we need to declare and pass around, which speeds up implementation and future debugging and improvements.

Okay, but these two pieces of code we have just seen are related to a specific activity. However, it is not uncommon to actually need to create custom structures, even though MQL5 already provides twelve predefined structures. At another time in the future, we will cover each one of these structures, Well, we have already discussed one of them briefly: the MqlDateTime. But that was only a brief introduction to what we can and will do later.

Now you may be wondering, dear reader: How do I implement something that uses a structure I created myself? Well, to answer this properly, we will create a new topic, as I do not want to mix subjects. Even though the answer is somewhat related to the current topic. Still, I want to keep things clear.

Working with Custom Structures

Although the answer to the question posed at the end of the previous topic is closely related, I want to separate things to make the material as didactic as possible. This is because when we use one of the twelve predefined structures in MQL5, we can implement things in a certain way. However, when we implement something using a structure we created ourselves, we often need to implement things differently. This is due to reasons explained in the previous article. And another reason we will discuss shortly.

But before that, let's understand something: 

A structure is a special data type that can contain any information.

This statement is very important to understand correctly. When you truly grasp what a structure actually is, it becomes much easier to understand how to implement and use it in your code. So, dear reader, understand that a structure is not different from a uchar, double, or any other type. Just as these types allow us to store and work with values contained in them, we can do the same using structures. This is somewhat similar to what we saw with unions, which are not truly a special data type, but rather a set of data sharing a common memory region.

To reinforce the idea that a structure is indeed a special data type, let us think about registration records. In a registration record, we may have various pieces of information such as first name, last name, address, profession, contacts, among others. All these data can be organized so that they share the same base format. This makes it possible to manipulate records in a very simple and practical way.

There are basically two ways to do this. The first is to create a series of variables to contain the information we want to store in a record. These same variables would be scattered throughout the code. Something like what can be seen below. Of course, this is just a hypothetical example.

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. void OnStart(void)
05. {
06.     string      Name,
07.                 Surname,
08.                 Address,
09.                 Position;
10.     double      Remuneration;
11.     datetime    Birth,
12.                 Hiring;
13. 
                   .
                   .
                   .
            Registration routines
                   .
                   .
                   .

Code 03

Unless you are creating an application whose purpose is to handle one single record, it is unlikely that any real code would look like what is shown in Code 03. However, you might indeed have code resembling Code 03, provided that the records are being stored on disk. But this is where some problems arise, for example, the order in which variables need to be stored. Yet this problem is not what truly interests us at the moment. The real issue is dealing with cross-references or relationships between data in different records.

Okay, so how could you handle this? The most common approach is to create arrays. Since we already talked about arrays, you might immediately think of creating something like what is shown below:

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. void OnStart(void)
05. {
06. #define def_Number_of_Records  10
07. 
08.     string      Name[def_Number_of_Records],
09.                 Surname[def_Number_of_Records],
10.                 Address[def_Number_of_Records],
11.                 Position[def_Number_of_Records];
12.     double      Remuneration[def_Number_of_Records];
13.     datetime    Birth[def_Number_of_Records],
14.                 Hiring[def_Number_of_Records];
15. 
                   .
                   .
                   .
            Registration routines
                   .
                   .
                   .

Code 04

Notice that now we can work with more than one record at the same time. However, although this technique works, it is rather clumsy. That is because, if we ever need to add or remove a field, we will face a lot of work for a simple change. And I'm only referring to the implementation, not the usage. But when looking at Code 04, and considering what we have discussed so far about structures, you may wonder: Can't we put everything from Code 04 into a structure? And that is exactly the point where things start to make sense. Because all that complexity seen in earlier code becomes something like what we see next.

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. void OnStart(void)
05. {
06. #define def_Number_of_Records  10
07. 
08.     struct st_Register
09.     {
10.         string      Name,
11.                     Surname,
12.                     Address,
13.                     Position;
14.         double      Remuneration;
15.         datetime    Birth,
16.                     Hiring;
17.     };
18.     
19.     st_Register Record[def_Number_of_Records];
20. 
                   .
                   .
                   .
            Registration routines
                   .
                   .
                   .

Code 05

Notice how naturally the idea emerged. We didn't need to create an entire application for the concept to take shape and make sense. But now - where things really get interesting - the structure declared on line eight of Code 05 will only be accessible within the OnStart procedure, or wherever it was declared. In this case, you must understand that the st_Register structure cannot be used to pass data to a function or procedure in the same way as in Code 02. You would need to adopt an approach very similar to that of Code 01 to make use of any information in the structure declared on line eight of Code 05.

In some cases, this is problematic. In others, it may be exactly what we need. In general, however, structures (unlike unions) are declared with a global scope. This allows us to transfer data in a more pleasant and practical way.

Anyone who has even superficially studied languages such as Python or JavaScript knows that it is very common to use something like what is shown below:

cmd_entry.pack(fill='x', side='left' , expand=1)

In this line of code valid both in Python and JavaScript, we have something that makes no sense to someone who uses MQL5, C, or C++. That's because when you see a value being assigned to a variable in these languages, you are indeed assigning a value to a variable present in the code, whether global or local. And if you search for that variable, you will find it in one of those scopes speaking in MQL5, C, or C++ terms.

However, if the same snippet is seen in Python or JavaScript, we are not actually assigning a value to a variable. I know this sounds completely illogical. But we are assigning a value to something similar to a structure. Because of this, we can assign values to any element in any order, which would be impossible in MQL5, for instance.

Yet nonetheless, if we look at this same code in Python or JavaScript, as something analogous to a structure in MQL5, everything starts to make sense. It would make passing values between functions and procedures much simpler and more pleasant, especially when many arguments need to be passed.

Now pay close attention, because this may make a difference in how you approach future problems. In some previous articles, we discussed the fact that we can overload functions and procedures. This allows us to handle different types or different numbers of arguments. But if, in some situations, we used structures to pass these values, many of the overloads that would otherwise be necessary would no longer be needed. This is because we can pass an arbitrary number of arguments simply by placing them inside a structure.

Pretty interesting, right? Indeed, there is no perfect, absolute way to do things. But there are approaches that are more or less interesting depending, of course, on each specific case.

So let's suppose the following: suppose Code 05 actually uses several functions and procedures, each requiring more or fewer arguments. You could use the approach seen in Code 01 and control each argument, but at the cost of possibly needing function overloading in multiple places. You could also adopt the approach seen in Code 02, but the cost would be the risk that some value may be modified without you noticing. However, there is a solution that, in many cases, is indeed adopted. This solution involves using a function when you want something to be modified, and using a procedure when you do not want modification, only consultation.

In any case, the structure declared on line eight of Code 05 would need to be moved out of local scope and placed in global scope, as shown in the hypothetical code below.

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. struct st_Register
05. {
06.     string      Name,
07.                 Surname,
08.                 Address,
09.                 Position;
10.     double      Remuneration;
11.     datetime    Birth,
12.                 Hiring;
13. };
14. //+------------------------------------------------------------------+
15. void OnStart(void)
16. {   
17.     st_Register Record[10];
18.     uchar       Counter = 0,
19.                 Index;
20. 
21.     Index = Counter;
22.     Counter += (NewRecord(Record[Counter], "Daniel", "Jose", "Brasil", "1971/03/30") ? 1 : 0);
23.     Record[Index] = UpdatePosition(Record[Index], "Chief Programmer");
24.     Index = Counter;
25.     Counter += (NewRecord(Record[Counter], "Edimarcos", "Alcantra", "Brasil", "1974/12/07") ? 1 : 0);
26.     Record[Index] = UpdatePosition(Record[Index], "Programmer");
27.     Index = Counter;
28.     Counter += (NewRecord(Record[Counter], "Carlos", "Almeida", "Brasil", "1985/11/15") ? 1 : 0);
29.     Record[Index] = UpdatePosition(Record[Index], "Junior Programmer");
30.     Index = Counter;
31.     Counter += (NewRecord(Record[Counter], "Yara", "Alves", "Brasil", "1978/07/25") ? 1 : 0);
32.     Record[Index] = UpdatePosition(Record[Index], "Accounting");
33. 
34.     Print("Number of records: ", Counter);
35.     ViewRecord(Record[3]);
36. }
37. //+------------------------------------------------------------------+
38. bool NewRecord(st_Register &arg, const string Name, const string Surname, const string Address, const string Birth)
39. {
40.     arg.Name = Name;
41.     arg.Surname = Surname;
42.     arg.Address = Address;
43.     arg.Birth = StringToTime(Birth);
44.     arg.Hiring = TimeLocal();
45. 
46.     return true;
47. }
48. //+------------------------------------------------------------------+
49. st_Register UpdatePosition(const st_Register &arg, const string NewPosition)
50. {
51.     st_Register info = arg;
52. 
53.     info.Position = NewPosition;
54. 
55.     return info;
56. }
57. //+------------------------------------------------------------------+
58. void ViewRecord(const st_Register &arg)
59. {
60.     PrintFormat("Collaborator: %s, %s\nPosition: %s\nBirth in: %s",
61.                     arg.Surname,
62.                     arg.Name,
63.                     arg.Position,
64.                     TimeToString(arg.Birth, TIME_DATE));
65. }
66. //+------------------------------------------------------------------+

Code 06

I know that Code 06 is somewhat subjective - something that many might find unlikely to be created. But even though it is only a hypothetical example, it contains many elements and concerns that appear in real-world code. When executed, it produces the following:

Figure 03

We have several things happening here. And although the code may seem complicated and difficult to understand to some, for those who have been studying and practicing what has been shown in the articles, this code is actually quite simple, clear, and straightforward. But since there are some elements that may seem odd, such as birth dates, I will provide a quick explanation of what is happening.

On line four, we declare our data structure. Note that it is declared in global scope. Because of this, it can be used outside the main code block - in this case, the OnStart procedure. Inside the main block, we have the declaration on line 17, which aims to create a list of information. In other words, our structure will indeed be treated as a special type of variable. After this, between lines 21 and 32, we add several records to our list. Remember that each value inside the array is like a completely distinct element. Yet, because we structure everything within a single block 9the structure declared on line 04) it feels as though each element were physically linked to the others, forming a sort of record sheet.

Thus, with each call between lines 21 and 32, we first attempt to add a new record to our array through the function on line 38. Notice that we are granting the function on line 38 the minimum privileges necessary. We only provide some information, while other values can be assigned directly by the routine itself, such as on line 44, where we take the current date and use it to set the hiring date. In reality, this is often how things are done, since the moment an employee record is created is indeed the moment they are hired. For this reason, we allow the function on line 38 to assign a value to the returned structure - a value that is not passed as an argument.

Of course, in real code there would be many checks to prevent errors during the registration phase. But since this code is purely didactic, such checks are unnecessary.

Since each record needs other data that are not passed directly as arguments to the NewRecord function on line 38, we use line 49 to update or assign some specific value to the already created record. Again, in real code there would be many checks here.

But the purpose of the function on line 49 is to show how we could return a value, in this case, the very structure created inside the function. This practice allows us finer control over when and where we change values in a structure.

Notice that unlike line 38, where the input structure is permanently modified, on line 49 we modify the structure and return it to the caller. It is up to the caller to determine where and how to use the returned value. And yes, dear reader, you can use this return value in many ways, as will be shown later.

Finally, we have a small procedure to display a record. It gives us the result shown in Figure 03.

Could we create the same Code 06, but in a way that has the work being done differently? Yes. And that is exactly what we will see in the code below:

001. //+------------------------------------------------------------------+
002. #property copyright "Daniel Jose"
003. //+------------------------------------------------------------------+
004. struct st_Register
005. {
006.     uchar       ID;
007.     string      Name,
008.                 Surname,
009.                 Address,
010.                 Position;
011.     double      Remuneration;
012.     datetime    Birth,
013.                 Hiring;
014. }gl_Record[10];
015. //+------------------------------------------------------------------+
016. void OnStart(void)
017. {
018.     uchar Index;
019. 
020.     ZeroMemory(gl_Record);
021. 
022.     NewCollaborator("Daniel", "Jose", "Brasil", "1971/03/30", "Chief Programmer");
023.     NewCollaborator("Edimarcos", "Alcantra", "Brasil", "1974/12/07", "Programmer");
024.     NewCollaborator("Carlos", "Almeida", "Brasil", "1985/11/15", "Junior Programmer");
025. 
026.     Index = GetNumberRecord();
027. 
028.     if (NewRecord(gl_Record[Index], "Yara", "Alves", "Brasil", "1978/07/25"))
029.     {
030.         gl_Record[Index].ID = Index + 1;
031.         gl_Record[Index] = UpdatePosition(gl_Record[Index], "Accounting");
032.     }
033. 
034.     Print("Number of records: ", GetNumberRecord());
035.     Print("--------------------");
036. 
037.     for(uchar c = 1; ViewRecord(c); c++)
038.         Print("********************");
039. }
040. //+------------------------------------------------------------------+
041. bool NewCollaborator(const string Name, const string Surname, const string Address, const string Birth, const string Position)
042. {
043.     st_Register info;
044.     uchar       Index = 0;
045. 
046.     if (!NewRecord(info, Name, Surname, Address, Birth))
047.         return false;
048. 
049.     info = UpdatePosition(info, Position);
050. 
051.     while (gl_Record[Index].ID)
052.     {
053.         if (Index >= gl_Record.Size()) return false;
054.         Index++;
055.     }
056. 
057.     info.ID = Index + 1;
058.     gl_Record[Index] = info;
059. 
060.     return true;
061. }
062. //+------------------------------------------------------------------+
063. bool NewRecord(st_Register &arg, const string Name, const string Surname, const string Address, const string Birth)
064. {
065.     arg.Name = Name;
066.     arg.Surname = Surname;
067.     arg.Address = Address;
068.     arg.Birth = StringToTime(Birth);
069.     arg.Hiring = TimeLocal();
070. 
071.     return true;
072. }
073. //+------------------------------------------------------------------+
074. st_Register UpdatePosition(const st_Register &arg, const string NewPosition)
075. {
076.     st_Register info = arg;
077. 
078.     info.Position = NewPosition;
079. 
080.     return info;
081. }
082. //+------------------------------------------------------------------+
083. uchar GetNumberRecord(void)
084. {
085.     uchar counter = 0;
086. 
087.     for (uchar c = 0; c < gl_Record.Size(); counter += (gl_Record[c].ID ? 1 : 0), c++);
088. 
089.     return counter;
090. }
091. //+------------------------------------------------------------------+
092. bool ViewRecord(const uchar ID)
093. {
094.     st_Register info;
095. 
096.     ZeroMemory(info);
097. 
098.     for (uchar c = 0; (c < gl_Record.Size()) && (!info.ID); c++)
099.         info = (gl_Record[c].ID == ID ? gl_Record[c] : info);
100. 
101.     if (info.ID)
102.         PrintFormat("Collaborator: %s, %s\nPosition: %s\nBirth in: %s",
103.                         info.Surname,
104.                         info.Name,
105.                         info.Position,
106.                         TimeToString(info.Birth, TIME_DATE));
107.     else
108.         Print("Record ID [", ID ,"] not found.");
109. 
110.     return (bool) info.ID;
111. }
112. //+------------------------------------------------------------------+

Code 07

Okay, now we really have something quite complex in our hands. That's because Code 07 is definitely more complicated, and I understand very little of what is being done here. (LOL) Well, deep down, Code 07 is actually very simple. To speak honestly, in my view, Code 07 is simpler than Code 06, even though they essentially do the same thing. However, I understand that some readers might perceive Code 07 as more complicated.

First, there is the fact that we now have a global variable declaration on line 14. And this makes anything seem more complex than would naturally be required. But before we discuss in detail what Code 07 is doing, let's look at the execution result. This can be seen in the image below:

Figure 04

Notice that all the records we are creating within the main block appear in this image, which is perfect, as it indicates that Code 07 is fulfilling its role. But how was this achieved, especially given the new elements appearing here? Well, dear reader, as mentioned in the previous article, structures represent a step between conventional programming and object-oriented programming. Yet, we are still very early in exploring what can truly be done with structures.

Now, notice that lines 22 to 32 are used to insert new records into the system. And this may be the key point: the way records are created between lines 22 and 24 is relatively simple to understand at first glance, unlike what happens between lines 26 and 32. There, we are performing the same work as the function on line 41, but entirely manually. The mere fact of doing it this way makes the code somewhat riskier. Even a small slip could cost hours of troubleshooting.

I encourage you to study Code 07 carefully in order to understand this practically. Compare the difficulty of producing a result with Code 06 to the difficulty of achieving the same result using Code 07. Understanding this will make it easier to follow what will be discussed in the next article.

However, perhaps the most complex part of Code 07 is the function on line 92. Let's analyze how it works. First, on line 94, we declare a temporary structure. Then, on line 96, we clear the memory region where the structure will be placed. This allows us to use the tests in the loop on line 98 to determine when the requested ID has been found.

The loop then concludes so that on line 101, we can check whether there is something to print in the MetaTrader 5 terminal. If the ID has been found, we print the content of the record structure. Otherwise, we print the message shown on line 108.

Since the return type of this function must be boolean, and the ID is a numeric value, we inform the compiler that we are aware of this fact. Thus, on line 110, we perform an explicit type conversion to return to the caller whether the search for the requested ID was successful. This return is important so that the loop on line 37 knows when to stop.

Final Thoughts

In this article, we saw how to use structures in relatively simple code. Although everything here is primarily didactic, we were able to build something that closely resembles a database. It is true that a real database would require numerous tests to prevent errors. However, I believe the main objective has been achieved: to demonstrate the concept of structures.

In other words, a structure is indeed a special type of variable. So, take the time to study and practice with the code provided in the attachments. While a line-by-line explanation of each piece of code was not provided, I believe it is not strictly necessary, as the content here is understandable for anyone who has been studying and practicing the material presented in the articles.

In the next article, we will explore how structures can be used in an even more practical and interesting way, making the coding process significantly simpler.