From Basic to Intermediate: Arrays and Strings (II)

Introduction

The content presented here is intended solely for educational purposes. Under no circumstances should the application be viewed for any purpose other than to learn and master the concepts presented.

In the previous article, From Basic to Intermediate: Arrays and Strings (I), we offered a brief introduction to strings and arrays, although that article was, in practice, almost entirely focused on strings. Everything covered there served only as a light and concise introduction to the topic. However, in order for us to move on to other subjects, we must first delve a bit deeper into the previous explanations. Therefore, the prerequisite for following and understanding the material presented in this article is having understood the content from the previous one.

In addition to this initial prerequisite, there are others that are equally important, such as understanding the IF statement and loop constructs, as well as having some knowledge of operators and variables. All of these topics should be familiar to you if you have studied the previous articles. If that is not the case, it is recommended that you review the earlier material to fully grasp what will be discussed here.

We can now begin to address the main topic of this article. Here is our new topic.

Custom Formatting

One of the most enjoyable aspects of programming in MQL5 is that we often don't need to worry about creating certain things from scratch. In fact, it is rare for the MQL5 standard library not to meet our needs. However, there are situations where it may not fully correspond to what we require or want to do. One such case is creating custom formatting to display certain types of information, whether in the terminal or within a graphical object we intend to use.

Since, at this stage, we won't yet be working with graphical objects, all of our tasks will be focused on using the standard output of MetaTrader 5, in other words, the terminal. I'll soon show you how to make things much more interesting. But for that moment to come, we first need to build a solid and consolidated understanding of some key features of a programming language.

Therefore, this initial content is not exclusively tied to MQL5. It can be applied in other programming languages as well, especially the parts dealing with programming concepts and rules. As mentioned at the beginning of this topic, there are scenarios where the MQL5 standard library won't be sufficient. But if you use the knowledge from the previous articles and a bit of logical thinking, you can implement many interesting things, even if, at first glance, we've only covered basic topics that may seem to lack a clear purpose.

The act of assigning a purpose to a particular piece of knowledge isn't, in itself, the most exciting part of being a programmer. The truly interesting part lies in the moment you begin to understand how things work and start thinking of ways to apply that knowledge to solve a specific problem. That's when the learning really gains meaning.

Now, I'd like you to consider the following problem. Suppose you want to view or print binary values in the MetaTrader 5 terminal. The reason doesn't matter, only the problem itself. So, you're faced with this challenge and immediately think: "Well, I can use string formatting to display binary values in the terminal". Perfect, that would be a solid first thought from a good programmer. However, a more experienced programmer would already know the answer. A beginner, on the other hand, might search the documentation for a way to format a string to display the binary representation of a numeric value and will find that such functionality doesn't exist. And it's at this moment that we need creativity combined with knowledge.

Since a built-in format for displaying binary values doesn't exist, we’ll need to create our own. If you're new to programming and only have the knowledge covered in the previous articles, you might assume this is impossible. But is it really, dear reader? Let's see if it can be done using only the information provided in earlier articles.

We know that there's a function in the standard library that allows us to format various types of data. All we need to do is create our own format to handle binary values. This is the easiest part, but there is a more difficult task. Unlike C or C++, which allow an unlimited number of arguments to be passed to a function, MQL5 does not offer this flexibility. As far as I know, C and C++ are the only languages that allow such functionality Assembly does too, but let's be honest, no one's crazy enough to write something in Assembly. Frankly, that would be a sign of madness.

Alright. The inability to pass an unlimited number of arguments to a function makes things seem more complicated at first. But we can break the problem down into manageable steps. Since we want to create a new format while still using an existing one from the MQL5 library, we can manipulate the data accordingly. How? Simple, dear reader. We create a function that returns a string containing the binary representation of a value. Since the MQL5 standard library can work with strings when formatting output, it becomes easy to define what will effectively be our custom format.

Thinking of it this way, it may seem like a difficult task. But you'll see just how simple it is by looking at the code below:

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. void OnStart(void)
05. {
06.     uchar value = 138;
07. 
08.     PrintFormat("The value %d in binary is written as: %s", value, BinaryToString(value));
09. }
10. //+------------------------------------------------------------------+
11. string BinaryToString(const uchar arg)
12. {
13.     return "--BINARY--";
14. }
15. //+------------------------------------------------------------------+

Code 01

When we run code 01, the result will be like this:

Figure 01

At this point you might think, "Wow, but this is not what we wanted to implement". Indeed, this is not what we wanted. But this serves to show you that we can build on what already exists and try to create something new. Notice that the text string found in line 13 is the same as in image 01. In other words, it works.

Now all we have to do is convert the value received by the function into binary representation. We then insert this representation into a string and return it. We do this in line 13.

How can we convert the resulting value into a string that contains only zeros and ones? It's all very simple, dear readers. For this we use a loop. There is one small point, however. But I will explain it directly in the code, which can be seen below:

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. void OnStart(void)
05. {
06.     uchar value = 138;
07. 
08.     PrintFormat("The value %d in binary is written as: %s", value, BinaryToString(value));
09. }
10. //+------------------------------------------------------------------+
11. string BinaryToString(const uchar arg)
12. {
13.     string  sz0 = "";
14.     uchar   mask = 0x80;
15. 
16.     while (mask)
17.     {
18.         sz0 = sz0 + ((arg & mask) == mask ? "1" : "0");
19.         mask = mask >> 1;
20.     }
21. 
22.     return sz0;
23. }
24. //+------------------------------------------------------------------+

Code 02

When you run this code, you'll see the following output in the MetaTrader 5 terminal:

Figure 02

It's absolutely perfect. In other words, with minimal and basic knowledge, we managed to accomplish the task. However, there's a small detail in this code. Based on the knowledge demonstrated up to this point, we cannot use just any type of data to convert it to its binary representation. This would not be possible without getting into more advanced programming concepts. That said, for simple cases where we're working with a single data type - in this case, uchar. We already have a function that works effectively. And as you can see, it's quite simple. Since everything being done here was already covered in previous articles, there's no need for me to explain how this function performs its task. However, there's one operator being used that we haven't yet discussed: the shift operator. Since it appears in the function, I believe it's good to explain how it works. That will be the focus of the next section. But before we move on, I'd like to draw your attention to one important detail: the 'mask' variable must have the same bit width as the 'arg' variable. This ensures a correct and accurate conversion and translation of values. As more advanced articles are published, we'll learn how to make the compiler handle this aspect for us. This will allow us to use of any data type, from the most complex to the simplest, without having to modify much in the function shown in Code 02. But that's a topic for the future. For now, let's learn how the shift operator works. So let's move on to a new topic.

The Shift Operator

Operators are generally self-explanatory. However, the shift operator can be a bit confusing for some people, as it is primarily used in computing contexts. As far as I know, it doesn't appear outside the fields of computer science or digital electronics. The shift operator, which you can observe on line 19 of Code 02, is highly practical and quite useful in a variety of cases. However, it is NOT a special operator as it merely serves as a shorthand for other operations, typically replacing two more verbose operators you're likely to encounter in some code. I say "two" because there are indeed two shift operators: one for shifting to the right, and one for shifting to the left. Many beginners confuse these shift operators with relational operators used to compare values, such as greater than or less than. This confusion usually stems from a lack of attention, since shift operators are composed of double arrows (either left or right), while relational operators use single arrows.

So, what exactly does the shift operator do? When used, it tells us how many bits to push a given value to the left or right. This may sound a bit complex at first, but it's actually much simpler than the operations it replaces. 

But what does that mean? I don't understand. If there's another way to express the same thing, why don't we just use that instead? That's a fair question. I can show you. In fact, that's precisely why I'm writing this section. I want you to understand that the same result can be achieved through different implementations. Saying two programmers came up with the exact same solution to a problem is highly unlikely. Not impossible, but definitely rare. That's because the way a problem is solved often depends on each programmer's level of knowledge and experience.

Now, let's return to our main point. To explain, in a clear and simple way how shift operators work, we're going to play around with the code shown below.

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. void OnStart(void)
05. {
06.     const uchar shift = 1;
07.     uchar   i0 = 153,
08.             i1,
09.             i2,
10.             i3,
11.             i4;
12. 
13.     i1 = i0 << shift;
14.     i2 = i0 * (uchar)MathPow(2, shift);
15.     i3 = i0 >> shift;
16.     i4 = i0 / (uchar)MathPow(2, shift);
17. 
18.     PrintFormat("Shift value          : %d\n" + 
19.                 "Original             : %s\t%d\n" +
20.                 "Shifted to the left  : %s\t%d\n" +
21.                 "Multiplication result: %s\t%d\n" +
22.                 "Shifted to the right : %s\t%d\n" +
23.                 "Division result      : %s\t%d"
24.                 , shift, BinaryToString(i0), i0, 
25.                          BinaryToString(i1), i1, 
26.                          BinaryToString(i2), i2, 
27.                          BinaryToString(i3), i3, 
28.                          BinaryToString(i4), i4
29.                 );
30. }
31. //+------------------------------------------------------------------+
32. string BinaryToString(const uchar arg)
33. {
34.     string  sz0 = "";
35.     uchar   mask = 0x80;
36. 
37.     while (mask)
38.     {
39.         sz0 = sz0 + ((arg & mask) == mask ? "1" : "0");
40.         mask = mask >> 1;
41.     }
42. 
43.     return sz0;
44. }
45. //+------------------------------------------------------------------+

Code 03

When you execute this code, you'll see the output shown in the image below:

Figure 03

Now pay close attention, dear reader. This Code 03, which at first might look quite confusing, is actually very clear and it fulfills that purpose effectively. But to fully understand it, you need to know what the MathPow function does. As it's part of the MQL5 standard library, I recommend checking the official documentation for more details. In short, it simply performs exponentiation. However, it returns a double value. Since our data is of type uchar, we explicitly cast the type. This will prevent the compiler from complaining about mismatched data types.

Now, note that in Image 03, we are displaying both the value and its binary representation on each line. This binary representation is important because it makes it clearer what happens to our original value. You might say that shifting a value to the left multiplies it by two, and shifting it to the right divides it by two. In a way, you're not wrong: that is, if we analyze only Figure 03. But what happens if we change the shift amount from one to three, for example?

Figure 04

Figure 04 shows what happens when we shift by three positions. In this case, you might notice something a bit strange. Shifting by three doesn't mean we're simply multiplying or dividing by two, it means we're working with powers of two. That is, we're dividing or multiplying by two raised to a given number. The correct result is achieved precisely through the use of the shift operator. Since performing this kind of operation (with exponentiation followed by multiplication or division) is more computationally expensive, it's preferable to use the shift operator instead.

However, when looking at the values in Figure 04, you may notice that part of the information disappears. This is not a bug in the program, dear reader. It's a result of data type limits. MQL5 is a strongly typed language, and when we exceed the limits of a type, some information is lost. There are various ways to avoid or work around this, but that's not our focus at the moment. We'll cover how to handle these cases at a more appropriate time. And trust me, in programming, there's always room for more learning.

I believe it's now clear how the shift operator works. With this understanding, Code 02 should now be easily understood. We can now move on to another subject, which is still somewhat connected to strings and arrays.

Pangram and Anagram

There's an interesting and sometimes even fun application that we can explore at this point, even with the limited material we've covered so far. I'm referring to pangrams. A pangram is a sentence that contains every letter of the alphabet, often used for testing purposes. Although it's a fairly old concept, it provides a highly didactic and engaging exercise. Perhaps the most well-known pangram of all is:

A QUICK BROWN FOX JUMPS OVER THE LAZY DOG 0123456789

This sentence was originally used to test old telegraph systems. It includes all the characters that might appear in a message. However, what made it even more famous was Microsoft's use of it in Word to preview how text would be printed. Back then, some people even claimed there were hidden messages in Microsoft tools, which made many users afraid to use computers.

But even within the context of what we're discussing right now, we can still play around with this kind of phrase. This will allows to learn more about strings and arrays. It's simple and quite enjoyable, and I believe you'll enjoy learning how it works.

It's not uncommon to hear people say we should use strong passwords, or even full phrases as passwords. But creating strong passwords or remembering them is often a difficult task. On the other hand, many suggest using specialized software to store passwords. Still, if those tools require a master password to access all the others, I personally don't find that particularly helpful.

However, even a beginner programmer who understands how basic functions and commands work can build a simple tool to generate a relatively strong password. Of course, the strength of the password will depend on the system and how it's designed. But with a bit of creativity, dear reader, you can definitely come up with something quite interesting, especially since the goal here is purely educational.

Let's start by considering the following: What is a password? Well, a password is essentially a set of printable characters stored in a string. So if we take a string and manipulate it, we can create a program that shuffles or encrypts the string. Since the goal is to create a strong password, this shuffling, which might be a form of encryption or simply an anagram, can effectively conceal, at least partially, the original password. That said, anagrams aren't the best for hiding sensitive information like a password that will be used elsewhere. In that case, shuffling or encryption is a better option, using an anagram or pangram as the base phrase. The key would be a sentence you can easily remember, and the final result would be the password you use.

To help you better understand what I'm describing, let's look at a simple and unpretentious piece of code. You can see it just below:

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. void OnStart(void)
05. {
06.     const string sz_Secret_Phrase = "Um pequeno jabuti xereta viu dez cegonhas felizes";
07. 
08.     Print("Result:\n", PassWord(sz_Secret_Phrase));
09. }
10. //+------------------------------------------------------------------+
11. string PassWord(const string szArg)
12. {
13.     const string szCodePhrase = "The quick brown fox jumps over the lazy dog";
14. 
15.     string  szPsw = "";
16.     uchar   pos;
17. 
18.     for (int c = 0; c < StringLen(szArg); c++)
19.     {
20.         pos = (uchar)(StringGetCharacter(szArg, c) % StringLen(szCodePhrase));
21.         szPsw = StringFormat("%s%c", szPsw, StringGetCharacter(szCodePhrase, pos));
22.     }
23. 
24.     return szPsw;
25. }
26. //+------------------------------------------------------------------+

Code 04

Looking at this code, you might be thinking: What on earth are you trying to do here? Relax, dear reader. What we're doing here may seem a bit crazy and nonsensical at first glance. But soon you'll understand exactly what I'm trying to show you. Consider the following: this code uses only what we've covered up to this point. Nothing new is introduced here. Everything can be fully understood if you've followed the previous articles and taken time to study the library functions used in this example. Everything here relies on features that can be easily found in the MQL5 documentation. Therefore, I won't go into detail about the functions used. However, when you run this code without making any modifications, you'll see the following output:

Figure 05

The region highlighted in Figure 05 represents the password you should use. But notice that it's somewhat flawed due to the presence of space characters. Typically, passwords do not contain spaces. So, we need to correct this. One very simple way to fix this is by removing the spaces and capitalizing the first letter. This is quite easy to achieve. After making this change to line 13 of code 04, we will get something similar to the code below:

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. void OnStart(void)
05. {
06.     const string sz_Secret_Phrase = "Um pequeno jabuti xereta viu dez cegonhas felizes";
07. 
08.     Print("Result:\n", PassWord(sz_Secret_Phrase));
09. }
10. //+------------------------------------------------------------------+
11. string PassWord(const string szArg)
12. {
13.     const string szCodePhrase = "TheQuickBrownFoxJumpsOverTheLazyDog";
14. 
15.     string  szPsw = "";
16.     uchar   pos;
17. 
18.     for (int c = 0; c < StringLen(szArg); c++)
19.     {
20.         pos = (uchar)(StringGetCharacter(szArg, c) % StringLen(szCodePhrase));
21.         szPsw = StringFormat("%s%c", szPsw, StringGetCharacter(szCodePhrase, pos));
22.     }
23. 
24.     return szPsw;
25. }
26. //+------------------------------------------------------------------+

Code 05

At this point, comparing code 05 and code 04, we can say: "Nothing has changed. This won't work. The result will be very similar to the previous one." But is it true? Let's find out. When code 05 is executed, the output will be as follows:

Figure 06

Wow! It changed, and changed a lot. It's completely different. It looks nothing like what was shown in Figure 05. Many people assume that to program something like this, you must be a math genius or a highly experienced developer with years of practice. However, what we see here is that even with just a basic understanding of programming, combined with a healthy dose of creativity, you can achieve very impressive and sometimes even surprising results. Now, if you find that this password is still too simple, even though it's quite long, we can fix that very easily. Keep in mind that the only difference between Code 04 and Code 05 is line 13. No other line was modified. So naturally, you might start to think: What if we added some extra symbols to line 13? Would that make the password stronger? Well, let's try that too, dear reader. So now, let's update line 13 to the version shown next.

    const string szCodePhrase = ")0The!1quick@2brown#3fox$4jumps%5over^6the&7lazy*8dog(9";

The output will be:

Figure 07

Isn't this a fun little project we're working on? And we are using very basic programming here. But we can make things a bit more refined, though this will require a slightly different approach.

This time, I'm going to do something a little different to help you understand a detail mentioned in the previous article. There, I noted that the string type does not exist in C or C++. However, in MQL5, this type does exist, specifically to spare us from relying on the programming approach used in C and C++. In C and C++, a string is essentially an array that contains a special value indicating where the string ends. Since MQL5 uses the same value for the same purpose, we can adjust the password-generating code so that it no longer relies on MQL5 standard library calls.

Now, I want you to pay close attention. We're about to use something in MQL5 that comes directly from C and C++. That's why I said MQL5 sits in a kind of middle ground, as I explained in the previous article. For now, I'll just give a brief introduction to this topic, which we'll explore more thoroughly in the next article.

Strings Are Arrays

The title of this section states exactly what happens under the hood in programming. I realize this may sound confusing at first. But in the next article, we'll dive deeper into this topic. For now, I want to leave you with this concept so you can begin looking into it early. Things are about to get a lot more complex, and quickly, once we start discussing this subject in more depth.

Taking advantage of this moment, we can update the code shown in the previous section to treat the string as an array. This might seem odd, but there are several advantages to approaching it this way from a programming standpoint.

Thus, Code 04, as seen earlier, becomes the version shown below:

01. //+------------------------------------------------------------------+
02. #property copyright "Daniel Jose"
03. //+------------------------------------------------------------------+
04. void OnStart(void)
05. {
06.     const string sz_Secret_Phrase = "Um pequeno jabuti xereta viu dez cegonhas felizes";
07. 
08.     Print("Result:\n", PassWord(sz_Secret_Phrase));
09. }
10. //+------------------------------------------------------------------+
11. string PassWord(const string szArg)
12. {
13.     const string szCodePhrase = "The quick brown fox jumps over the lazy dog";
14. 
15.     uchar   psw[],
16.             pos;
17. 
18.     ArrayResize(psw, StringLen(szArg));
19.     for (int c = 0; szArg[c]; c++)
20.     {
21.         pos = (uchar)(szArg[c] % StringLen(szCodePhrase));
22.         psw[c] = (uchar)szCodePhrase[pos];
23.     }
24.     return CharArrayToString(psw);
25. }
26. //+------------------------------------------------------------------+

Code 06

When you run Code 06, the output will be identical to what was shown in Figure 05. However, if you modify line 13, as described earlier, you'll notice that the results remain consistent with the previous ones. That said, Code 06 brings some advantages, as we're now working with an array-based system instead of using a string directly. The benefits of this approach will be better explained in the next article, where I’ll go into more detail about how to take advantage of this type of modeling. And believe it or not, dear reader, this code is still simple enough that any well-instructed beginner with a dedicated study habit can understand it, without me even having to explain how it works.

In any case, I will explain this code thoroughly in the next article. I'm only showing it to you now to ensure the material doesn't start to pile up. Don't assume that just because it looks simple and easy, you can leave it for later.

Final Considerations

In this article, which, in my opinion, was quite fun, we created something practical, with a real purpose beyond just applying theoretical concepts. I know some of the points we touched on may seem daunting at first. But that shouldn't be a reason to give up. Quite the opposite. What I want to demonstrate here is that even a beginner programmer, equipped with creativity and a bit of cleverness, can build and manipulate text data to create something that many believe requires years of programming experience.

I believe that having seen these concepts applied using only basic knowledge - as everything here is based on what was covered in earlier articles - will make you more engaged and less passive. Hopefully, it's now clear that the material you've learned so far is useful and relevant. That's it. So, enjoy exploring the attached files. See you in the next article where we'll talk about ARRAYS.