Using text files for storing input parameters of Expert Advisors, indicators and scripts

Introduction

When developing and using various trading tools, we sometimes come across the cases when standard means of inputs creation via extern and input modifiers are not sufficient enough. Although we have a universal solution to satisfy all our needs, it sometimes turns out to be rather cumbersome and inflexible. Let's consider the following cases as examples.

• Using standard tools to create the parameters that are to be rarely changed or are not to be changed at all (i.e., order magic number or slippage value).

• If only one data source should be selected among a dozen of such indicators as RSI, WPR etc. in the indicator or EA properties, the parameters of each of them should be created and made available in the properties window despite the fact that we need the parameters of only one of them. Another example of this kind is a weekly EA operation schedule. There is no much sense in storing the data that was relevant on Monday and Tuesday (or will be relevant on Friday) on Wednesday. But we still have to do this because the entire schedule is to be located in the inputs. In other words, description of dynamically created objects in the parameters is quite a difficult and inefficient task.

• If we need to place a note, comment or header into the list of tool's properties, we have to create a new string property with an appropriate content. This may be inconvenient at times. 
  

How to store parameters in text files

Plain text files can be used as an addition to the conventional method of creating and storing the input parameters. You can put anything you want in them, besides they can be easily edited and moved. Their structure can be arranged similar to INI files. For example, here is how an integer type array saved in a text file may look:

/*array size*/
{array_size},5
/*array*/
{array_init},0,1,2,3,4

In the example above, the "anchor" or the "section name" is written at the beginning of the string followed by the section content separated by comma. Any unique string of characters may serve as an "anchor". Such file is generated and saved in the terminal "sandbox". Next, we open this file in read-only mode (as a CSV file) in the initialization block of an indicator, EA or script.

When it is time to read the saved array, we search for the "anchor" in the file using the known name {array_size}. We should move the file pointer to the beginning of the file by calling FileSeek(handle,0,SEEK_SET). Now, let's search for the next "anchor" using a known name {array_init}. When it is found, we simply read the strings the necessary number of times transforming them according to the array type (in this case, transformation to integer is performed). ConfigFiles.mqh include file contains a few simple functions implementing the "anchor" search and data reading.

In other words, an object that we save in the file should generally be described by the "anchor" with the following string containing comma-separated data as required by CSV format. You can use as much "anchors" and following data strings as necessary, but keep in mind that one "anchor" corresponds only to one string.

Comments as well as various notes and instructions can be written to a file in almost any form and at any place. However, there is one obvious requirement: the text should not disrupt the "anchor" – data sequence. Another desirable condition – any large text should be located at the end of the file to simplify the search for the "anchors".

Below you can see the possible arrangement for storing the EA operation schedule mentioned above:

….......
{d0},0,0,22
{d1},0,0,22
{d2},1,0,22
…........
{d6},0,0,22

Here, the "anchor" name reflects the days of weeks using a certain number (for example, {d1} stands for the day number one, etc.). The "anchor" name is followed by a bool type value that defines if the EA should trade on that day or not (in our case, it does not trade on d1). If there is no trading, the following values may be ignored but they are left nevertheless. Finally, the last two values of integer type stand for the EA operation start and end in hours.

At the opening of a daily candle, the EA reads the operation schedule for the new day. Of course, the data string for each day can be changed and supplemented by any other values depending on the operation logic. Thus, only the current day data is stored in memory.

You can set any "anchor" name but I recommend using common sense. There is no point in assigning abstract names to "anchors". Instead, they should convey some meaning while remaining unique. Please keep in mind that data search is performed by names. Depending on a particular implementation, a single file may have several "anchors" of the same name.

Let's examine the code fragments taken from the fully operational indicator. The indicator needs data on several currency pairs for proper operation. So, it requests data by timer and then processes it according to its logic (the indicator logic is of no importance for us here). Please remember that brokers sometimes add various suffixes and prefixes to symbol names (for example, EURUSD may be turned into #.EURUSD.ch). This should be taken into account, so that the EA is able to refer to other symbols correctly. The sequence of our actions is as follows.

1. Create a text file broker.cfg with the following content: 

[PREFIX_SUFFIX],#.,.ch

2. Create a text file <indicator_name>.cfg with the following content:

/*Number of currency pairs with the "anchor" string [CHARTNAMES] */
[CHARTCOUNT],7
/*Names of the currency pairs, whose charts are used by the indicator to read the data*/
[CHARTNAMES],USDCAD,AUDCAD,NZDCAD,GBPCAD,EURCAD,CADCHF,CADJPY
/*Position of a base currency in a pair (it is CAD in this case) – first or second*/
[DIRECT],0,0,0,0,0,1,1

3. Place both files into the "sandbox".

4. Define several auxiliary functions in the indicator code (or include ConfigFiles.mqh file):

// Open file by a specified name like a CSV file and return its handle
   int __OpenConfigFile(string name)
     {
      int h= FileOpen(name,FILE_READ|FILE_SHARE_READ|FILE_CSV,',');
      if(h == INVALID_HANDLE) PrintFormat("Invalid filename %s",name);
      return (h);
     }

//+-------------------------------------------------------------------------------------+
//| The function reads one iRes value of long type in the section containing            |
//| strSec "anchor" name in the handle file. If successful,                             |
//| true is returned, otherwise – false.                                                |
//+-------------------------------------------------------------------------------------+
   bool _ReadIntInConfigSection(string strSec,int handle,long &iRes)
     {
      if(!FileSeek(handle,0,SEEK_SET) ) return (false);
      string s;
      while(!FileIsEnding(handle))
        {
         if(StringCompare(FileReadString(handle),strSec)==0)
           {
            iRes=StringToInteger(FileReadString(handle));
            return (true);
           }
        }
      return (false);
     }

//+-------------------------------------------------------------------------------------+
//| The function reads sArray having count size in the section containing               |
//| strSec "anchor" name in the handle file. If successful,                             |
//| true is returned, otherwise – false.                                                |
//+-------------------------------------------------------------------------------------+
   bool _ReadStringArrayInConfigSection(string strSec,int handle,string &sArray[],int count)
     {
      if(!FileSeek(handle,0,SEEK_SET) ) return (false);
      while(!FileIsEnding(handle))
        {
         if(StringCompare(FileReadString(handle),strSec)==0)
           {
            ArrayResize(sArray,count);
            for(int i=0; i<count; i++) sArray[i]=FileReadString(handle);
            return (true);
           }
        }
      return (false);
     }

//+-------------------------------------------------------------------------------------+
//| The function reads bArray of bool type having count size in the section containing  |
//| strSec "anchor" name in the handle file. If successful,                             |
//| true is returned, otherwise – false.                                                |
//+-------------------------------------------------------------------------------------+
   bool _ReadBoolArrayInConfigSection(string strSec,int handle,bool &bArray[],int count)
     {
      string sArray[];
      if(!_ReadStringArrayInConfigSection(strSec, handle, sArray, count) ) return (false);
      ArrayResize(bArray,count);
      for(int i=0; i<count; i++)
        {
         bArray[i]=(bool)StringToInteger(sArray[i]);
        }
      return (true);
     }

Besides, include the following code into the indicator:

   …..
   input string strBrokerFname  = "broker.cfg";       // Name of the file containing the broker data
   input string strIndiPreFname = "some_name.cfg";    // Name of the file containing the indicator settings
   …..

   string strName[];         // Array with symbol names ([CHARTNAMES])
   int    iCount;            // Number of currency pairs
   bool   bDir[];            // "Sequence" array ([DIRECT])

   …..
   int OnInit(void) 
     {

      string prefix,suffix;     // Prefix and suffix. Local variables necessary for 
                                // initialization but used only once.
      prefix= ""; suffix = "";
      int h = _OpenConfigFile(strBrokerFname);
      // Read data on prefix and suffix from the configuration file. If errors 
      // are fixed, continue with default values.
      if(h!=INVALID_HANDLE) 
        {
         if(!_GotoConfigSection("[PREFIX_SUFFIX]",h)) 
           {
            PrintFormat("Error in config file %s",strBrokerFname);
              } else {
            prefix = FileReadString(h);
            suffix = FileReadString(h);
           }
         FileClose(h);
        }
      ….
      // Read the indicator settings. 
      if((h=__OpenConfigFile(strIndiPreFname))==INVALID_HANDLE)
         return (INIT_FAILED);

      // read the number of symbols
      if(!_ReadIntInConfigSection("CHARTCOUNT]",h,iCount)) 
        {
         FileClose(h);
         return (INIT_FAILED);
        }
      // read the array with symbol names after reading their number
      if(!_ReadStringArrayInConfigSection("[CHARTNAMES]",h,strName,iCount)) 
        {
         FileClose(h);
         return (INIT_FAILED);
        }

      // transform symbol names to the appropriate look
      for(int i=0; i<iCount; i++) 
        {
         strName[i]=prefix+strName[i]+suffix;
        }

      // read the array of bool type parameters
      if(!_ReadBoolArrayInConfigSection("[DIRECT]",h,bDir,iCount)) 
        {
         FileClose(h);
         return (INIT_FAILED);
        }
      ….
      return(INIT_SUCCEEDED);
     }

Two dynamic arrays and two critical local variables have been initialized during the code execution. The generated broker.cfg file will come in handy quite often relieving you from the necessity to constantly enter a prefix and suffix manually.

Possible areas of application. Drawbacks and alternatives

Apart from the already mentioned cases, the proposed method allows you to conveniently manage several instances of an indicator or an EA working on different currency pairs. It can also be used for similar tasks requiring a single "control center". The method can also come in handy when you only need to edit or replace a text file for remote configuration without accessing the terminal itself. It is not uncommon when FTP is the only means of access to a PC with the terminal installed.

Another possible application area arises when a trader manages a dozen of terminals located in different places (maybe even in different countries). Simply prepare the configuration file once and send it to the "sandboxes" of all terminals. If we go back to the previously mentioned example involving the schedule, it is possible to arrange its delivery once per week to ensure synchronous operation of EAs on all PCs. The indicator, whose code fragments were shown above, works on 28 pairs and is managed synchronously by the two mentioned files only.

Yet another interesting application area involves storing a variable in a file and a property area simultaneously. In this case, you are able to implement a sort of a reading priority logic. To illustrate this, let's consider a pseudocode fragment based on magic number initialization:

…..
extern int Magic=0;
…..
int OnInit(void) 
  {
   …..
   if(Magic==0) 
     {
      // This means that a user has not initialized Magic and
      // the default value has remained. Let's search for Magic variable in a file then
      …...
     }
   if(Magic==0) 
     {
      // Still zero. Thus, there is no such variable in a file
      // Process the current situation,
      // exit with an error
      return (INIT_FAILED);
     }

This example reveals another advantage of this method. In case the file is used for managing multiple indicator/EA instances, the method allows performing more thorough fine-tuning of each instance separately while bypassing centralized configuration from the file. 

In order to remain objective, let's describe the disadvantages of the method.

The first one is low speed. However, this may not be much of an issue if you apply the method only during an indicator/EA initialization or periodically (for example, at the opening of a day candle).

The second one – thorough and careful approach to supporting documentation is critical. It can be very difficult to demonstrate to traders how to manage tools customized this way. This is another reason to store rarely changed settings in such files. Of course, the utmost attention should be paid to the preparation and editing text files themselves. Errors at this stage may cause serious financial consequences.

Now, let's examine alternative solutions for the mentioned tasks.

The first one is using INI files. This is a reliable data storage method. INI files are used when you do not want to write to the registry. Their structure is clear and easily comprehensible. Below you can see our first example (operation schedule) written in INI file:

….......
[d0]
Allowed=0
BeginWork=0
EndWork=22
.........
[d2]
Allowed=1
BeginWork=0
EndWork=22

In all other aspects, working with INI files is similar as well – they should be placed to the "sandbox" and their operation speed is also rather low (though a bit better than that of CSV files). Just like with CSV files, it is recommended to use them during an initialization or at a new candle opening. Keep in mind though that you cannot place comments in an INI file text the way you like since INI files have their own rules regarding comments. Anyway, you will most probably face no problems with supporting documentation caused by the file format.

The disadvantage of this format is that you need to include third-party libraries. MQL languages provide no means for working with INI files directly, therefore you will have to import them from kernell32.dll. You can find several libraries providing such possibility on MQL5 website, so there is no point in attaching another one here. Such libraries are simple since only two functions are imported. However, they may still contain errors. Besides, you can never be sure the entire construction is compatible with Linux. If you are not afraid of third-party libraries, then you are welcome to use INI files on a par with CSV ones.

Apart from INI files, there are other possible solutions. Let's examine them in brief.

1. Using the registry. In my view, this is the most questionable solution, since the registry is critical for the operating system's normal operation. Therefore, it would be strategically incorrect to allow scripts and indicators to write into it.
2. Using the database. This is a reliable method of storing any volume of data. But does an EA or an indicator really need such amount of data? In most cases, the answer is "no". The functionality of databases is clearly redundant for achieving our goals. However, databases are still handy in case you really have to store multiple gigabytes of data.
3. Using XML. In fact, these are the same text files with different syntax that you need to master beforehand. In addition, you should develop a library (or download a ready-made one) to work with XML files. The most difficult task is preparing supporting documentation at the end of the work. I believe, the results are hardly worth the trouble in this case.

Conclusion

In conclusion, I would like to note that text files are already widely used in various aspects of trading, for example, in trade copying mechanisms. MetaTrader terminals generate and use a variety of different text files, including configuration files, various logs, emails and templates.

The article has presented an attempt to apply common CSV text files in configuring trading tools – EAs, indicators and scripts. I have used a few simple examples to illustrate new opportunities provided by such files. Also, some alternatives and detected drawbacks have been analyzed.

Anyway, inputs storage method should always be selected according to developer's current tasks. The efficiency of such a choice is one of the factors demonstrating the developer's expertise.