From Basic to Intermediate: Union (I)

Introduction

The materials presented here are intended for educational purposes only. Under no circumstances should the application be viewed for any purpose other than to learn and master the concepts presented.

In the previous article From Basic to Intermediate: Array (IV), we explored a very cool and extremely interesting concept. Although many consider it to be an advanced topic, in my humble opinion, it is something every beginner in programming should know. That's because, when used correctly, the very concept introduced in the previous article can quite literally open the whole world of possibilities. With it, we can do things that would otherwise be very difficult or even impossible to achieve.

Moreover, that same concept is also used in another type of context that we will cover at a more appropriate time. So as not to cause anyone undue anxiety, here's a small tip: study and practice what was covered in the previous article extensively. It is important to understand that knowledge thoroughly. Without it, absolutely nothing from this point forward will make sense. Everything that follows will seem like magic.

Well, perhaps saying that nothing will make sense unless you've understood the previous article is a bit of an exaggeration on my part. However, that doesn't change the fact that the previous article is the most important one published up to now. At least for those who genuinely want to become good programmers. And for those who aspire to be able to do anything with a programming language. The concept introduced in the previous article isn't limited to MQL5 - it applies to any programming language, particularly when it comes to making proper use of computational resources.

OK, so before we get started, we need to discuss the prerequisites for this current article. Although some instructors might think I'm exaggerating, in my view, unless you've at least superficially understood what was presented in the previous article, it will be very unlikely that you'll be able to follow what we're going to do here. I'm not saying you won't understand it. But it will definitely be much more difficult to keep up with the explanations.

So, the previous article serves as a kind of watershed moment. On one side, we have all the basic programming material; now, we're jumping to slightly more advanced material. And as the article title suggests, we're going to talk about UNION. But not "union" in the general sense of the word. We're referring to the term UNION as it appears in certain programming languages. And as usual, we'll start a new topic that will make the implementation and coding process much more enjoyable and fun.

The Birth of UNION

In the previous article, we implemented the following code:

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. #include "Tutorial\File 01.mqh"
05. //+------------------------------------------------------------------+
06. void OnStart(void)
07. {
08.     const uint  ui = 0xCADA5169;
09.     ushort      us = 0x43BC;
10.     uchar       uc = B'01011101';
11. 
12.     uchar       Infos[],
13.                 counter = 0;
14.     uint        start,
15.                 number;
16. 
17.     PrintFormat("Translation personal.\n" +
18.                 "FUNCTION: [%s]\n" +
19.                 "ui => 0x%s\n" + 
20.                 "us => 0x%s\n" +
21.                 "uc => B'%s'\n",
22.                 __FUNCTION__,
23.                 ValueToString(ui, FORMAT_HEX),
24.                 ValueToString(us, FORMAT_HEX),
25.                 ValueToString(uc, FORMAT_BINARY)
26.                );
27.     
28.     number = sizeof(ui) + 1;
29.     start = Infos.Size();
30.     ArrayResize(Infos, start + number);
31.     Infos[counter++] = sizeof(ui);
32.     Infos[counter++] = (uchar)(ui >> 24);
33.     Infos[counter++] = (uchar)(ui >> 16);
34.     Infos[counter++] = (uchar)(ui >> 8);
35.     Infos[counter++] = (uchar)(ui & 0xFF);
36. 
37.     number = sizeof(us) + 1;
38.     start = Infos.Size();
39.     ArrayResize(Infos, start + number);
40.     Infos[counter++] = sizeof(us);
41.     Infos[counter++] = (uchar)(us >> 8);
42.     Infos[counter++] = (uchar)(us & 0xFF);
43. 
44.     number = sizeof(uc) + 1;
45.     start = Infos.Size();
46.     ArrayResize(Infos, start + number);
47.     Infos[counter++] = sizeof(uc);
48.     Infos[counter++] = (uc);
49. 
50.     Print("******************");
51.     PrintFormat("The Infos block contains %d bytes.", Infos.Size());
52.     ArrayPrint(Infos);
53.     Print("******************");
54. 
55.     Procedure(Infos);
56. 
57.     ArrayFree(Infos);
58. }
59. //+------------------------------------------------------------------+
60. void Procedure(const uchar &arg[])
61. {
62.     Print("Translation personal.\n" +
63.           "FUNCTION: ", __FUNCTION__);
64. 
65.     ulong value;
66. 
67.     for (uchar c = 0; c < arg.Size(); )
68.     {
69.         value = 0;
70.         for (uchar j = arg[c++], i = 0; (c < arg.Size()) && (i < j); i++, c++)
71.             value = (value << 8) | arg[c];
72.         Print("0x", ValueToString(value, FORMAT_HEX), " B'", ValueToString(value, FORMAT_BINARY), "'");
73.     }
74. }
75. //+------------------------------------------------------------------+

Code 01

When we run Code 01, we get the result shown below:

Figure 01

In Figure 01, we can see that the values defined on lines 8, 9, and 10 are passed into the procedure located on line 60. However, when we look at the data type expected by that procedure, we notice that not individual values are passed but an array. At first glance, this might not seem to make any sense. Many people, looking at the procedure on line 60, would expect to see operations involving an array. But that's not quite what's happening here. What we're actually seeing is a transcription or translation of the values contained within the array to reconstruct the values that were passed.

Notice that we have no way of knowing the name or the type of the variable used when calling the procedure. That's because there is no reference to that kind of information. All we know is the number of elements and the value of each element that would correspond to each variable.

Many programmers (even some with good experience) consider this kind of approach completely unacceptable, since there's no direct mapping between a value and a named variable in the source code. However, they often forget that for the CPU, the name of a variable is irrelevant. All the CPU sees is a sequence of numbers. That's it. It has no idea what the name of a variable or constant is. That kind of information is entirely irrelevant to it.

Therefore, the memory model created by what we see in Code 01 looks something like the representation shown below:

Figure 02

The concept we're discussing can be quite confusing, especially when we later examine another concept that looks very similar to this one.

In Figure 02, we see the values highlighted in Figure 01, along with some markers. These markers appear in green and serve the following purposes:

Here begins a new value, and it consists of this many elements.

The blue rectangles represent each element block. That is, if we didn't use an array and instead used discrete variables, we would need six separate variables, since there are six blue rectangles in Figure 02. However, we also have red rectangles. They represent each element within the array. Since there are ten elements, we see ten red rectangles. We could also have five red rectangles if each element consisted of two values.

However, due to fragmentation concerns (in my view, this topic is too advanced to explain here) we use a minimum size for each element. This helps avoid fragmentation, even though it slightly slows down the decoding process later. This process is performed by the loop on line 70, though many might think it is handled by the loop on line 67. In fact, the decoding happens in the loop on line 70.

OK. I believe that up to this point, this part was not too difficult. But now, look closely at Figure 02 and pause to think: Is there a way to read the blue rectangles without necessarily going through the red rectangles? Or, put differently: is it possible, just by looking at the blue rectangle, to obtain the values from the red rectangles? That would make our lives significantly easier. It would speed up both the encoding process between lines 28 and 48 and simplify decoding, since we would be directly from the blue rectangles. This is unlike what's currently happening in Code 01, where we're breaking down each blue rectangle in order to retrieve what would become the red elements. These red elements are stored in the array.

Indeed, dear reader, this idea born from this concept is what gave rise to what we know as a union. When we use a union, we create a shared memory structure that divides the blue rectangles into units of different sizes. The size of each unit or element depends on the purpose and goal of the code or application being developed. But once a union has been defined, it allows the programmer to control each part of a larger block in a completely individualized manner. This concept is widely used in C and C++ code, where unions help us manipulate entire chains of elements in a very simple, smooth, and safe way.

To understand how a union works, let's start with something simple before we return to modifying Code 01. So, let's examine the code shown below:

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. void OnStart(void)
05. {
06.     union un_01
07.     {
08.         ulong   u64_bits;
09.         uint    u32_bits[2];
10.     }info;
11. 
12.     uint tmp;
13. 
14.     info.u64_bits = 0xA1B2C3D4E5F6789A;
15. 
16.     PrintFormat("Before modification: 0x%I64X", info.u64_bits);
17. 
18.     tmp = info.u32_bits[0];
19.     info.u32_bits[0] = info.u32_bits[1];
20.     info.u32_bits[1] = tmp;
21. 
22.     PrintFormat("After modification : 0x%I64X", info.u64_bits);
23. }
24. //+------------------------------------------------------------------+

Code 02

Notice how one concept naturally leads to another. Without understanding arrays, as explained previously, you won't be able to grasp what's being done at this point in the code. And without understanding the previous article, what you'll see here won't make any sense at all. But now, I believe it's the right moment to explain what a union is and what its practical purpose is.

When you run Code 02, you'll see in the MetaTrader 5 terminal the image shown below:

Figure 03

This is the key point of unions. Notice how it seems almost magical. This is just the first level of what we can do. But let's understand exactly what's happening in Code 02. First, a union must be declared, as seen on line six. In other words, we use the reserved keyword union, followed by an identifier that serves as the union's name. There's a difference here between how unions are declared in MQL5 and how they're declared in C or C++. In MQL5, it is not possible to create an anonymous union. This means that the identifier following the union keyword must exist.

In languages like C and C++, when the identifier is omitted, we get what's known as an anonymous union. The downside of an anonymous union is that you can't reference it outside its declaration block. We'll see an example of this in another code snippet shortly. But for now, let's stay focused on Code 02. As you can see, once the union has been declared and given a name, it starts with an opening brace { and ends with a closing brace }. Everything inside this block is part of the union and shares the same memory region.

Now comes the crucial part we've been waiting for. Everything in memory must be thought of in terms of bytes. ABSOLUTELY EVERYTHING. So, in order to correctly define a union, you need to think about how memory will be divided in terms of bytes. Take another look at Figure 02. There, we have 10 bytes represented. How you group these (though "group" may not be the most accurate term) is what will guide you in creating your union. The union defined on line six will occupy eight bytes of memory. Its largest member is the variable u64_bits.

What about the u32_bits variable, which is an array? Wouldn't it also take up another eight bytes, since each of its elements occupies four bytes? So shouldn't the union occupy 16 bytes in total? No, dear reader. The union will in fact occupy only eight bytes. That's because u64_bits and u32_bits are sharing the same memory space.

I know this may seem extremely confusing at first. So let's take it slowly because things will only get more complicated if we skip over any steps in the explanation.

The goal of Code 02 is precisely to perform a swap between one part of memory and another. In the end, we want to rotate the information stored in memory. To do this, we need a temporary variable. It is declared on line 12. An important note: this temporary variable must have a size (in bytes) that is equal to or greater than the smallest variable in the union. Typically, we use the same type, which ensures the correct size is maintained. This is important to achieving the intended result.

Then, on line 14, we initialize the union. Pay close attention here. The actual variable whose memory region is used is declared on line 10. But because we cannot directly reference that memory region (as it may contain multiple variables), we must tell the compiler which member variable of the union we want to access. You can use any member that belongs to the union. If you do this properly, the compiler will understand and assign the correct value, updating the memory region accordingly.

To understand this better, let's go back once again to Figure 02. Imagine that the entire image actually is a union - and that's not far from the truth. (We'll revisit this idea in more detail later). Now imagine that each red rectangle has a name. If you want to access one of them, all you need to do is tell the compiler the name of that rectangle, and it will read from or write to that specific part of memory.

Pretty cool, right? But let's take it one step at a time. First, you need to deal with code 02. Once we assign a value to the u64_bits member, the entire memory region named info now holds the value specified on line 14.

To confirm this, we use line 16 to display the memory content. This prints our first output line in the terminal. Now here's the most interesting part: We want to swap the values within that memory, essentially creating a new value in the same memory space. Using the memory sharing provided by the union, we can do this quickly and easily.

The first step is to use our temporary variable to store one of the values. It's done on line 18. On line 19, we assign the value at index 1 of the array to index 0. At this point, our memory is a bit of a mess as it contains only part of the original value. To complete the process, we use line 20 to move the original value that was in index 0 into index 1. The swap is now complete, and line 22 prints the result - this is the second line shown in Figure 03.

If you thought this was wild, get ready for an even more interesting example where we mirror the entire content in a simple and efficient way. Take a look at Code 03, shown below:

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. void OnStart(void)
05. {
06.     union un_01
07.     {
08.         ulong   u64_bits;
09.         uchar   u8_bits[sizeof(ulong)];
10.     };
11. 
12.     un_01 info;
13. 
14.     info.u64_bits = 0xA1B2C3D4E5F6789A;
15. 
16.     PrintFormat("The region is composed of %d bytes", sizeof(info));
17. 
18.     PrintFormat("Before modification: 0x%I64X", info.u64_bits);
19. 
20.     for (uchar i = 0, j = sizeof(info) - 1, tmp; i < j; i++, j--)
21.     {
22.         tmp = info.u8_bits[i];
23.         info.u8_bits[i] = info.u8_bits[j];
24.         info.u8_bits[j] = tmp;
25.     }
26. 
27.     PrintFormat("After modification : 0x%I64X", info.u64_bits);
28. }
29. //+------------------------------------------------------------------+

Code 03

This Code 03 is even more interesting than the previous one. And when you run it, you'll see in the MetaTrader 5 terminal what is shown in Figure 04 below.

Figure 04

Man, what a crazy thing. But really cool and very interesting. I had been trying to do this sort of thing myself: struggling, sweating, and fighting through it just to get it working. Then along comes this approach, showing how we can do it in such a simple, easy, and practical way. Really impressive. I loved it! But now I have a few questions.

Let's go step by step so I can understand better. First question: What exactly is going on in line 9? When working with unions, it's common to include arrays to make it easier to access specific parts of the shared memory. There are other ways to do this without arrays, but we'll look into that later. Most of the time, you'll want access to the entire memory region so you can manipulate it properly. So it's not uncommon to see something like what appears on line 9. Although, in real-world code, the declaration might look a bit different. The principle and purpose remain the same: to create a way to access each element of the memory block inside the union.

Next question: What is that strange thing that is declared on line 12? That makes no sense to me." Line 12 is a perfect demonstration of something we discussed in Code 02. Remember we mentioned that anonymous unions are not allowed in MQL5? Well, because we defined the union type on line 6 using an identifier un_01, we're now able to declare variables of that type, like the one on line 12. You can have multiple variables, each holding different union values, all based on the same identifier. That's because when you define a special type (and a union is a special type), you can reuse its identifier throughout your code, just like any other type.

Just keep in mind that this identifier follows the same scope and visibility rules as any other variable or constant. This was discussed in earlier articles. But there's something important to remember here: a union will ALWAYS be a variable, it cannot be a constant. Even if you point it to a constant value, the union itself will always be treated as a variable. A special type of variable, much like a string, but still a variable.

That's why these articles are structured the way they are, so that you understand certain foundational concepts before seeing them applied. Without understanding the difference between a variable and a constant, it would be difficult to explain what we're discussing here. Another very important point: an array inside a union WILL NEVER, UNDER ANY CIRCUMSTANCES, be of the dynamic type. It will always be static.

Therefore, there's no point in trying to create a massive union using a dynamic array. If you attempt this, the compiler won't understand what you're trying to do.

One last detail you may not have noticed yet: how is the loop on line 20 able to mirror the contents of the memory region? For the mirroring to work, you must use a counter that iterates only halfway through the memory region. Since we always use an even number of elements, this is easy to do. Thus, the loop on line 20 can indeed mirror any value of a discrete type — as long as you make the necessary adjustments (not to the loop itself, but to the declaration of the block within the union on line six). Naturally, you'll also need to adjust the value being declared on line 14. But aside from that, no further changes to the code would be required.

For example: If you want to mirror a variable of type int, or any 32-bit value, you only need to change the type declarations on lines eight and nine from ulong to int. After doing that, simply update the value declared on line 14 to the desired one — and that's it. The code will now be capable of mirroring an int instead of a ulong. Simple as that.

There is an even simpler way to achieve this. However, I haven't yet introduced another concept we can use here in MQL5. So, the method explained above is, for now, the simplest approach available.

Before we wrap up this article, let's explore one final thing we can do with unions. This is related to their use in functions and procedures. To demonstrate this, we'll take code 03 and modify it so that the loop is first moved into a function. Then, we'll apply the same logic in the form of a procedure. Either way, the goal is for the function or procedure to perform the mirroring for us, and then we'll display the result in the main routine.

Yes, there are various ways to do this. But here, we'll use a didactic approach, as the goal is to show how we might use unions in a broader context. Let's begin with the code that, in my view, is the simplest to understand. That's because we'll use an implementation very similar to what we saw in code 03.

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. union un_01
05. {
06.     ulong u64_bits;
07.     uchar u8_bits[sizeof(ulong)];
08. };
09. //+------------------------------------------------------------------+
10. void OnStart(void)
11. {
12.     un_01 info;
13. 
14.     info.u64_bits = 0xA1B2C3D4E5F6789A;
15. 
16.     PrintFormat("The region is composed of %d bytes", sizeof(info));
17. 
18.     PrintFormat("Before modification: 0x%I64X", info.u64_bits);
19. 
20.     Swap(info);
21. 
22.     PrintFormat("After modification : 0x%I64X", info.u64_bits);
23. }
24. //+------------------------------------------------------------------+
25. void Swap(un_01 &info)
26. {
27.     for (uchar i = 0, j = sizeof(info) - 1, tmp; i < j; i++, j--)
28.     {
29.         tmp = info.u8_bits[i];
30.         info.u8_bits[i] = info.u8_bits[j];
31.         info.u8_bits[j] = tmp;
32.     }
33. }
34. //+------------------------------------------------------------------+

Code 04

Code 04 is quite simple to understand. However, you need to pay attention to a few important details. The first is that the union is no longer local - it is now global. This was done specifically to grant the procedure declared on line 25 access to the special type we defined on line four. Without making the union global, it would be impossible to use the special type we declared as un_01 in the parameter definition on line 25. Notice that all we did here was move the code that was previously inside the main routine into a separate procedure. And instead of the loop on line 20 of code 03, we use a call to our new procedure on the same line. Essentially, we've just made public a block of code that was previously private. I truly believe that you can understand Code 04 without any difficulty. That said, let's now look at a different scenario where, instead of using a procedure, we use a function. This example is shown below.

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. union un_01
05. {
06.     ulong u64_bits;
07.     uchar u8_bits[sizeof(ulong)];
08. };
09. //+------------------------------------------------------------------+
10. void OnStart(void)
11. {
12.     un_01 info;
13. 
14.     info.u64_bits = 0xA1B2C3D4E5F6789A;
15. 
16.     PrintFormat("The region is composed of %d bytes", sizeof(info));
17. 
18.     PrintFormat("Before modification: 0x%I64X", info.u64_bits);
19. 
20.     PrintFormat("After modification : 0x%I64X", Swap(info).u64_bits);
21. }
22. //+------------------------------------------------------------------+
23. un_01 Swap(const un_01 &arg)
24. {
25.     un_01 info = arg;
26. 
27.     for (uchar i = 0, j = sizeof(info) - 1, tmp; i < j; i++, j--)
28.     {
29.         tmp = info.u8_bits[i];
30.         info.u8_bits[i] = info.u8_bits[j];
31.         info.u8_bits[j] = tmp;
32.     }
33. 
34.     return info;
35. }
36. //+------------------------------------------------------------------+

Code 05

Well, things are a bit more complex here, but only because this might be a new concept for you at this stage. Note that the code is very similar to what we saw earlier when we used a procedure. However, if you look closely at line 20 across the last three examples, the version shown in Code 05 may be the most difficult for programmers with limited experience. But there's no reason to panic. After all, we';re simply working with a function declared on line 23. First, let's highlight something interesting. In both Code 03 and Code 04, the contents of the variable declared on line 12 are inevitably modified. That's a fact. However, in Code 05, the contents of that same variable declared on line 12 are not modified.

How can that be? Aren't we performing the mirroring operation to produce the result shown in Image 04? The mirroring is indeed happening. And in all three of these last examples, the result will match what you see in Figure 04. But when I say the variable on line 12 isn't modified, I mean exactly that — it remains unchanged. You can verify this by noting that, on line 23, we pass the value as a constant reference.

Now things may seem even more confusing. Earlier, I mentioned that unions cannot be used with constants, and now I'm saying we can. Confusing, isn't it? Alright, perhaps I didn't explain it clearly. Or maybe there's some confusion between assignment and declaration. The declaration on line 23 defines the parameter as constant. This means the variable passed to it cannot be modified within the function. However, for the function to operate, we still need a modifiable variable. It is precisely what we create on line 25. That variable is modified, and its result is returned on line 34.

This is the point where many people may get confused. If Swap is a function in Code 05, and it returns a variable, shouldn't we assign that return value to another variable before using it? It depends, dear reader. However, since a function is a type of variable, as discussed in a previous article, we can use the special type declared on line four to access the returned data directly.

This is only possible because the function is explicitly returning that specific type. If the return value were a discrete type, the implementation shown on line 20 wouldn't be feasible. In that case, we would need another mechanism to achieve the same result. But since that's a topic that could cause unnecessary confusion at this point, I'll leave it for another time. There are ways to accomplish it.

Final Thoughts

In this article, we began exploring what a union is. We experimented with the first few practical scenarios where a union might be applied. However, what we've seen here is just the basics, part of a larger set of concepts and techniques that we'll continue to develop in future articles. So here's a golden tip for you, dear reader. Practice and study thoroughly everything you've seen here.

In the attachment, you'll find the main code examples discussed in this article. And in the next article, we'll dive even deeper into the foundations of MQL5 programming. See you soon!