Enhancing the Quality of the Code with the Help of Unit Test

Андрей | 2 August, 2011

Introduction

While programming in MQL4, I had a wide experience dealing with some finished MQL4 programs and creating several dozens of my own ones. As a result, I came to the conclusion that MQL4 is a very favourable environment for creating low-quality programs. The reasons are listed below:

  1. MetaTrader 4 has no built-in debugger. Searching for errors can sometimes be quite a troublesome process.
  2. MQL4 has no such exceptions handling facilities as in C++ or Java.
  3. MQL4 programs are often created in a rush with much commitment to an idea than to a code quality.

All that leads to the code low quality and this fact in its turn results in the following issues:

  1. Errors in advisors operation, wrong operation algorithm (especially critical for real accounts).
  2. Slow execution. Very slow optimization.
  3. Bad processing of error cases. An expert advisor can turn out to be non-operable.

I'd like to note that all that was said before does not concern long-term experienced MQL4 programmers. Skilful programmers find ways to create a high-quality code.

As far as my primary job is connected with software quality testing, I was interested in any materials concerning MQL4 programs testing and debugging. But the number of the articles concerning this issue that I managed to find was small. Therefore, I'd like to describe here one of the ways to improve the quality of the programs. In case this topic proves to be interesting, other issues can be considered in the next articles.


Some Theory Concerning Quality

So, if we google a little, we can find out that:

Quality is a complex of production properties and characteristics that gives this production the possibility to satisfy conditional or assumed needs.

As far as software is concerned, we may consider the program to be of a good quality in case it complies with a customer's needs and fulfills the functions imposed on it correctly.

Two types of activity are usually needed to make a good quality program:

Quality assurance is a complicated issue. It concerns plenty of tasks beginning from programmer's comfortable working place creation up to the implementation of complicated business processes. We won't touch this issue yet. Let's talk about Quality Control.

The more attention we pay to the quality control, the higher is the chance that our program will work like a charm. Theoretically, the quality control (or testing, in other words) must be performed at each stage of development:

  1. Technical specification testing - it is impossible to develop a normally operating program based on an incorrect technical specification.
  2. Source code review. Searching for faults, ineffective code, coding rules violation, evident errors.
  3. Testing of the program's distinct functions in the automatic mode (Unit Testing).
  4. Testing of the whole finished program in the manual mode. A human (tester) checks if the program is working correctly.
  5. Program testing in the automatic mode (Automated Testing). That is when the robots test the program quality on their own. It sounds like a utopia but sometimes it works.
  6. Testing of a program by a customer.

etc. There are quite a lot of testing types...

And Unit Testing is among the most of interesting ones.


Some Theory Concerning Unit Testing

Google gives us the follwing definition of Unit Tests. Unit Testing - is a method of the application validation in which a programmer checks distinct units (blocks) of a source code for their usability in the rest of the program. Unit is the smallest part of a program that is suitable for testing. In application languages (including MQL4) a distinct function can be considered as a unit.

In most cases Unit Testing is done automatically. In other words, a program is created which calls an examined function with different parameters and then this program creates a report indicating if the values brought back by the function were true or not.

Unit Tests may be extremely useful because of the following reasons:

  1. In case a failure is detected, you can find its roots easily as only one function is tested. In case a failure in the whole application is detected, you will have to spend some additional time to find the function that is causing a trouble.
  2. It is quite easy to check if a defect is eliminated or not. Just run automatic Unit Test once more. There is no need to restart the whole application. For example, there can be some errors appearing on some definite rare occasions that can hardly be reconstructed. Unit tests eliminate this problem.
  3. You can optimize function code easily without any worries that something will go wrong. Unit test can always show if a function continues working normally or not.
  4. Also, you can detect problems that do not manifest themselves at once but can appear at a customer's place and require a lot of hours of searching and debugging.
  5. Modern Test Driven approach can be used, when a Unit Test is created at first and then the function is developed. The function is developed until the Unit test is passed. I tried this approach in one of the C++ applications for the first time and it proved to be good. I felt genuine delight as I was fully confident in the functions' usability at the end of their creation, and their further exploitation in the program was faultless.

Let's see how it looks like. Suppose, that we have created the function for square-rooting:
y=sqrt(x)

Therefore, we shall create another function for out test that will work according to the following algorithm:


We can create a testing function before creation of a main one. Therefore, we shall determine the requirements the created function must comply with. That is our use of the Test Driven approach. And we shall be able to use the function in our main program assuredly only after our Unit Test showed a faultless work.

But one question still remains open: how should we select a set of testing parameters for a tested function? Of course, it is necessary to use all possible values but in almost all cases it is impossible or too labor-intensive. A brand new article can be written on the test values selection. Here I'll try to give some general tips:

  1. It is necessary to use not only correct data but also the data that leads to errors, as we have to check not only if the function fulfills imposed duties but also how good it is at errors processing.
  2. We need to use boundary values. For example, if the values' range is from 0 up to 100, the values that equal to both 0 and 100 should be used. If the input data consists of lines, blank line and the line with a maximum length should be tried.
  3. The values that go beyond the permitted marks should be used. If we look at the example from the previous point, the values 101 and -1 should be used, the value max+1 should be used for the line.
  4. We should try to break the multitude of all possible values for subsets of equivalent ones (equivalence classes) for which the function's way of behaviour is similar. One value should be selected for every class. For example, there is no point in checking both sqrt(4) and sqrt(9). It is much more interesting to check sqrt(4) and sqrt(5), as in the latter case the function will bring the irrational value back, while in the first case it will be integral one.
  5. In case the function has branches (if, switch), we should assure that each of them is processed by the Unit Test.

I'll try to show this in the next chapter using a definite example.


Some Practice Concerning Unit Test Creation

Let's set a training goal! Suppose, our task is to develop the library which has the function accepting two arrays at the entry. The function deletes from the first array the elements that are absent in the second one. As a result, the first array is a subset of the second one.

Let's determine our function's prototype:

void CreateSubset(int & a1[], int a2[]);

We'll try to use Test Driven approach for the function development. Let's determine a set of test data. We should mark several input data equivalence classes to achieve this:

  1. Both arrays are empty.
  2. A1 is empty, A2 contains the elements.
  3. A1 contains the elements, A2 is empty.
  4. Both contain similar set of the elements and have similar size.
  5. A1 contains the elements that are not present in A2.
  6. Part of the elements in A1 are present in A2, A2 part is contained in A1 (both multitudes have an intersection).
  7. All A1 elements are present in A2, but A2 size is bigger.
  8. A small part of A1 elements is present in A2. Besides, the elements are scattered all over an array.
  9. A small part of A1 elements is present in A2. Besides, the elements are concentrated at an array leader.
  10. A small part of A1 elements is present in A2. Besides, the elements are concentrated at an array's end.

In case our function works well for all 10 cases, we may be absolutely sure that the experts that use this function will not suffer from its imperfection. But we should understand that it is impossible to test something for 100% and some possible latent defects can always remain.

I've created a small mql4unit library for convenience. I included there the functions that are necessary for unit tests:

//-------------------------------------------------------------------+

//Current test conditions are kept by the global variables
//-------------------------------------------------------------------+
int tests_passed;    //Number of successful tests
int tests_failed;    //Number of unsuccessful tests
int tests_total;     //Total number of tests

string test_name;    //Test name

//-------------------------------------------------------------------+
//The function initializes test environment for one test
//-------------------------------------------------------------------+
void UnitTestStart(string TestName)
{

   test_name = TestName;
   tests_passed = 0;
   tests_failed = 0;
   tests_total = 0;
   Print("*--------------------------------------------------*");

   Print("Starting unit test execution ", test_name);
}

//-------------------------------------------------------------------+
//the function is called at the end of the test. Brings back true if all the tests
//are successful. Otherwise - False.
//-------------------------------------------------------------------+
bool UnitTestEnd()
{
   if (tests_failed == 0)

   {
      Print("HURRAY!!! ", test_name, " PASSED. ", tests_passed, " tests are successful.");
   }
   else
   {

      Print(":((( ", test_name, " FAILED. ", tests_passed,"/",tests_total, " tests are successful.");   
   }
   Print("*--------------------------------------------------*");
}


//-------------------------------------------------------------------+
//The function executes the test for two arrays of int type
//Brings back true, if the arrays are equal
//-------------------------------------------------------------------+
bool TestIntArray(int actual[], int expected[]){

   tests_total++;
   //Comparing arrays' sizes
   if (ArraySize(actual) != ArraySize(expected))
   {
      Print("Test #", tests_total," ERROR. Array size ", ArraySize(actual), " instead of ", ArraySize(expected));

      tests_failed++;
      return(false);      
   }
   //Then comparing element by element
   for (int i=0; i<ArraySize(actual);i++)

   {
      if (actual[i]!=expected[i]){
         Print("Test #", tests_total," ERROR. Element value #",i,"=", actual, " instead of ", expected);
         tests_failed++;

         return(false);
      }
   }
   //If all the elements are equal, the test is passed
   Print("Test #", tests_total," OK: Passed!");  

   tests_passed++;
   return(true);
}

Let's create "mytests" test script with an empty body of our function. Create test function in it and describe all unit tests in it.
bool Test()
{
   UnitTestStart("CreateSubset function testing");
   Print("1. Both arrays are empty.");

   int a1_1[], a1_2[];
   int result_1[]; //Waiting for an empty array as a result of the function execution
   CreateSubset(a1_1, a1_2);
   TestIntArray(a1_1, result_1);
   
   Print("2. A1 is empty, A2 contains the elements");

   int a2_1[], a2_2[] = {1,2,3};
   int result_2[]; //Waiting for an empty array as a result of the function execution
   CreateSubset(a2_1, a2_2);

   TestIntArray(a2_1, result_2);

   Print("3. A1 contains the elements, A2 is empty");
   int a3_1[] = {1,2,3}, a3_2[];

   int result_3[]; //Waiting for an empty array as a result of the function execution
   CreateSubset(a3_1, a3_2);
   TestIntArray(a3_1, result_3);

   Print("4. Both contain similar set of the elements and have similar size");
   int a4_1[] = {1,2,3}, a4_2[] = {1,2,3};

   int result_4[] = {1,2,3}; //Waiting for an unchanged array as a result of the function execution
   CreateSubset(a4_1, a4_2);
   TestIntArray(a4_1, result_4);

   Print("5. A1 contains the elements that are not present in A2");

   int a5_1[] = {4,5,6}, a5_2[] = {1,2,3};
   int result_5[]; //Waiting for an empty array as a result of the function execution

   CreateSubset(a5_1, a5_2);
   TestIntArray(a5_1, result_5);
   
   Print("6. Part of the elements in A1 are present in A2, A2 part is contained in A1 (both multitudes have an intersection)");
   int a6_1[] = {1,2,3,4,5,6,7,8,9,10}, a6_2[] = {3,5,7,9,11,13,15};

   int result_6[] = {3,5,7,9}; //Waiting for arrays intersection as a result of the function execution
   CreateSubset(a6_1, a6_2);
   TestIntArray(a6_1, result_6);

   
   Print("7. All A1 elements are present in A2, but A2 size is bigger");
   int a7_1[] = {3,4,5}, a7_2[] = {1,2,3,4,5,6,7,8,9,10};

   int result_7[] = {3,4,5}; //Waiting for arrays intersection as a result of the function execution
   CreateSubset(a7_1, a7_2);
   TestIntArray(a7_1, result_7);
   

   Print("8. A small part of A1 elements is present in A2. Besides, the elements are scattered all over an array.");
   int a8_1[] = {1,2,3,4,5,6,7,8,9,10}, a8_2[] = {2,5,9};

   int result_8[] = {2,5,9}; //Waiting for arrays intersection as a result of the function execution
   CreateSubset(a8_1, a8_2);
   TestIntArray(a8_1, result_8);
   

   Print("9. A small part of A1 elements is present in A2. Besides, the elements are concentrated at an array leader.");
   int a9_1[] = {1,2,3,4,5,6,7,8,9,10}, a9_2[] = {1,2,3};

   int result_9[] = {1,2,3}; //Waiting for arrays intersection as a result of the function execution
   CreateSubset(a9_1, a9_2);
   TestIntArray(a9_1, result_9);

   Print("10. A small part of A1 elements is present in A2. Besides, the elements are concentrated at an array's end.");

   int a10_1[] = {1,2,3,4,5,6,7,8,9,10}, a10_2[] = {8,9,10};

   int result_10[] = {8,9,10}; //Waiting for arrays intersection as a result of the function execution
   CreateSubset(a10_1, a10_2);
   TestIntArray(a10_1, result_10);
   

   return (UnitTestEnd());
}

In order to execute the Unit Test we have to call the Test function at the main function and run the script.

Let's execute our test.


As we can see, the results are disappointing. That is not surprising, as the function is not ready at all. But nevertheless! 4 tests out of 10 are passed successfully. That means that theoretically we could have even missed the fact that the function is empty because in some cases it would have acted normally.

In other words, there can be such a subset of input data for which an incorrect function works correctly. If a programmer would have applied only such a test data that leads to success, the empty function could easily have gone to a customer.

Now, let's create the CreateSubset function itself. We shall not discuss efficiency and beauty of this function here.

void CreateSubset(int & a1[], int a2[]){
   int i=0;

   while(i<ArraySize(a1)){
      bool b_exist = false;
      for (int j=0; j<ArraySize(a2);j++){

         if (a1[i] == a2[j]) b_exist = true;
      }
      if (!b_exist){
         for (j=i; j<ArraySize(a1)-1;j++){
            a1[j] = a1[j+1];   

         }
         ArrayResize(a1, ArraySize(a1)-1);
      }else{
         i++;
      }
   }
}
Let's run the test again:


The function can be run from anywhere. It can be determined inside an expert and run during initialization. In case a separate module is processed, one or several test functions can be determined inside it and called from a script. Here we may fantasize a little.

Of course, the ideal variant would be to have possibility to run a unit test right after the library compilation but I cannot understand yet if this can be done in MQL4. Most likely not. If you know how this can be done, please, write me.

Every time after we run the test we can sigh with relief and be sure that everything works as it should.


A Few Comments

  1. It may seem that writing tests only consumes time. But I assure you that the time devoted to unit tests development will pay over and above.
  2. Of course, it is not worth developing Unit Tests for all functions. The balance should be kept between a function's importance, failure probability and amount of a code inside the function. For instance, there is no need to write a test for the simplest function consisting of a couple of lines.
  3. You can do anything you like inside unit tests: open/close orders, use indicators, graphic objects etc. Your actions are not limited here.


And Finally the Last Thing

I hope this material will be useful for you. I will gladly answer all your questions. Also, I'm open for any suggestions concerning the possible ways to improve this article and writing new ones.

I wish you all good luck and faultless codes!