Download MetaTrader 5

Grouped File Operations

26 August 2008, 09:30
MetaQuotes Software Corp.
0
834

Introduction

It is not a problem to read or write one file. Even if you use WinAPI for this, as described in article File Operations via WinAPI. But what should we do if we don't know the exact name of the file, only its location in a certain folder and its extension? Of course, we can enter the necessary name manually, as a parameter, but what should we do if there are ten or more such files? We need a method of grouped processing the files of the same type in the given folder. This task can be efficiently solved using functions FindFirstFile(), FindNextFile() and FindClose() included in kernel32.dll.


Function FindFirstFile()

The function description is given in msdn at: http://msdn.microsoft.com/en-us/library/aa364418(VS.85).aspx.

HANDLE WINAPI FindFirstFile(
  __in   LPCTSTR lpFileName,
  __out  LPWIN32_FIND_DATA lpFindFileData
);

If follows from this description that the function returns the descriptor of the file found and meeting the search requirements. The search condition is specified in variable lpFileName that contains the path to search for the file and a possible name of the file. This function is convenient, since we can specify the search by mask, for example, find a file by mask "C:\folder\*.txt". The function will return the very first file found in the folder of "C:\folder" and having the extension of txt.

The returned result of the function is of the 'int' type in MQL4. To pass the input parameter, we can use the 'string' type. Now we have to sort out about what to pass to this function as the second parameter and how to process that parameter later. This function will be imported approximately like this:

#import "kernel32.dll"
int  FindFirstFileA(string path, .some second parameter);
#import

Here we can see the already known library of kernel32.dll. However, the function name is specified as FindFirstFileA(), not as FindFirstFile(). This comes from fact that many functions in this library have two versions: for working with strings in Unicode, the letter 'W' (FindFirstFileW) is added to the name, while for working with ANSI the letter 'A' (FindFirstFileA) is added.

Now we have to sort out the function's second parameter that is described as:

lpFindFileData [out] - A pointer to the WIN32_FIND_DATA structure that receives information about a found file or directory.

It means that it is the pointer to a certain structure of WIN32_FIND_DATA. In this case, structure is a certain area in the PC RAM. The pointer to this area (address) is passed to the function. We can allocate memory in MQL4 using a data array. The pointer is specified with the '&' character. We have only to get to know the size of the necessary memory in bytes to pass the pointer at. Below is the structure description.

typedef struct _WIN32_FIND_DATA {
  DWORD dwFileAttributes;
  FILETIME ftCreationTime;
  FILETIME ftLastAccessTime;
  FILETIME ftLastWriteTime;
  DWORD nFileSizeHigh;
  DWORD nFileSizeLow;
  DWORD dwReserved0;
  DWORD dwReserved1;
  TCHAR cFileName[MAX_PATH];
  TCHAR cAlternateFileName[14];
} WIN32_FIND_DATA, 

In MQL4, there are no types DWORD, TCHAR or FILETIME. DWORD is known to occupy 4 bytes, like int in MQL4, TCHAR has an internal representation of one byte. To calculate the total size of structure WIN32_FIND_DATA in bytes, we have only to sort out what FILETIME is.

typedef struct _FILETIME {
  DWORD dwLowDateTime;
  DWORD dwHighDateTime;
} FILETIME

FILETIME turns out to contain two DWORDs, which means that DWORD makes 8 bytes. Let's tabulate all this:

Type Size in Bytes
DWORD 4
TCHAR 1
FILETIME 8

Now we can calculate the size of structure WIN32_FIND_DATA and visualize what and where can be found in it.

Type
Size in Bytes Note
dwFileAttributes 4File attributes
ftCreationTime 8File/folder creation time
ftLastAccessTime 8Last accessed time
ftLastWriteTime 8Last write time
nFileSizeHigh 4 Maximum size in bytes
nFileSizeLow 4 Minimum size in bytes
dwReserved0 4 It is usually not defined and not used
dwReserved1 4 It is reserved for the future
cFileName[MAX_PATH] 260 (MAX_PATH = 260)
File name
cAlternateFileName[14] 14Alternative name in the 8.3 format

So the total size of the structure is: 4 + 8 + 8 + 8 + 4 + 4 + 4 +4 + 260 +14 = 318 bytes.


As you can see from the figure above, the file name starts with the 45th byte, the preceding 44 bytes containing various auxiliary information. It is necessary to pass to function FindFirstFile() some structure in MQL4 sized as 318 bytes, as the second parameter. It would be most convenient to use an array of the 'int' type, which would have a size not less than the required one. Divide 318 by 4 (since the internal representation of the 'int' type is 4 bytes), obtain 79.5, round it to the nearest larger integer, and see that we need an array of 80 elements.

The function import itself will now appear as follows:

#import "kernel32.dll"
int  FindFirstFileA(string path, int & answer[]);
#import

Here we are using the version of the function with the letter 'A' at the end of the name, FindFirstFileA(), for ANSI coding. The 'answer' array is passed by a link and serves for being filled with the structure of WIN32_FIND_DATA. An exemplary call:

   int win32_DATA[80];
   int handle = FindFirstFileA(TerminalPath() + "\experts\*.mq4",win32_DATA);

Functions FindNextFileA() and FindClose()

Function FindNextFileA() receives, as the first parameter, the 'handle' of the file preliminarily obtained by function FindFirstFileA() or by another, earlier call to function FindNextFileA(). The second parameter is the same. Function FindClose() just closes the search. This is why the full appearance of importing data of functions will be as follows:

#import "kernel32.dll"
int  FindFirstFileA(string path, int & answer[]);
bool FindNextFileA(int handle, int & answer[]);
bool FindClose(int handle);
#import

Now we have just to learn how to extract the name of a file written to array 'answer[]'.

Getting the File Name

The file name is contained by the array starting from the 45th through the 304th byte. The 'int' type contains 4 bytes, so each array element contains 4 characters, if we assume the array to be filled out with characters. So, in order to refer to the first character in the file name, we should skip 44/4 = 11 elements of the 'answer[]' array. The file name is inside the chain of 65 (260/4=65) array elements, starting from the 'answer[11]' (indexing starts with zero) and ending with 'answer[76]'.

Thus, we can get from array 'answer[]' the file name in blocks by 4 characters in each. The 'int' number represents a sequence of 32 bits that, in their turn, represent 4 blocks by 8 bits in each.

The youngest byte is to the right, the oldest byte is to the left. Bits are numbered in the ascending order, i.e., bits from the 1st one to the 8th one make the youngest byte. We can extract the necessary bytes using bitwise operations. To get the value of the youngest byte, we should fill with zero values all bits, from the 9th through the 32nd one. This is performed using logic operation AND.

int a = 234565;
int b = a & 0x000000FF;

Here 0x000000FF is the 32-bit integer that has zero values of all places, starting from the 9th one, while the places from 1 through 8 are equal to one. Thus, the obtained number b will contain only one (the youngest) byte of number a. We will turn the obtained byte (the character code) into a one-character string using function CharToStr().

Ok, we have obtained the first character. How should we obtain the next one? Very easily: We make a bitwise 8-bit shift to the right, and the second bit replaces the youngest bit. Then we will apply the already known operation of logic AND.

int a = 234565;
int b = (a >>8) & 0x000000FF;

As you can guess, the third byte will be obtained by shifting by 16 bits, while the oldest will be obtained by shifting by 24 bits. Thus, we can extract 4 characters from one element of the array of the 'int' type. Below is shown how the first 4 characters in the file name are obtained from the 'answer[]' array:

   string text="";
   
   int pos = 11;
   int curr = answer[pos];
      {
      text = text + CharToStr(curr & 0x000000FF)
         +CharToStr(curr >> 8 & 0x000000FF)
         +CharToStr(curr >> 16 & 0x000000FF)
         +CharToStr(curr >> 24 & 0x000000FF);
      }
    Print("text = ", text);

Let's create a separate function that returns a text string from the passed array named 'buffer'.

//+------------------------------------------------------------------+
//|  read text from buffer                                           |
//+------------------------------------------------------------------+ 
string bufferToString(int buffer[])
   {
   string text="";
   
   int pos = 10;
   for (int i=0; i<64; i++)
      {
      pos++;
      int curr = buffer[pos];
      text = text + CharToStr(curr & 0x000000FF)
         +CharToStr(curr >> 8 & 0x000000FF)
         +CharToStr(curr >> 16 & 0x000000FF)
         +CharToStr(curr >> 24 & 0x000000FF);
      }
   return (text);
   }

Thus, the problem of obtaining the file name from the structure is solved.

Getting the List of All Expert Advisors with Their Source Codes

A simple script demonstrates the features of the above functions:

//+------------------------------------------------------------------+
//|                                                CheckFindFile.mq4 |
//|                      Copyright © 2008, MetaQuotes Software Corp. |
//|                                       http://www.metaquotes.net/ |
//+------------------------------------------------------------------+
#property copyright "Copyright © 2008, MetaQuotes Software Corp."
#property link      "http://www.metaquotes.net/"
 
#property show_inputs
 
#import "kernel32.dll"
int  FindFirstFileA(string path, int& answer[]);
bool FindNextFileA(int handle, int& answer[]);
bool FindClose(int handle);
#import
 
 
//+------------------------------------------------------------------+
//| script program start function                                    |
//+------------------------------------------------------------------+
int start()
  {
//----
   int win32_DATA[79];
   
   int handle = FindFirstFileA(TerminalPath() + "\experts\*.mq4",win32_DATA);
   Print(bufferToString(win32_DATA));
   ArrayInitialize(win32_DATA,0);
 
   while (FindNextFileA(handle,win32_DATA))
      {
      Print(bufferToString(win32_DATA));
      ArrayInitialize(win32_DATA,0);
      }
 
   if (handle>0) FindClose(handle);
   
//----
   return(0);
  }
  
//+------------------------------------------------------------------+
//|  read text from buffer                                           |
//+------------------------------------------------------------------+ 
string bufferToString(int buffer[])
   {
   string text="";
   
   int pos = 10;
   for (int i=0; i<64; i++)
      {
      pos++;
      int curr = buffer[pos];
      text = text + CharToStr(curr & 0x000000FF)
         +CharToStr(curr >> 8 & 0x000000FF)
         +CharToStr(curr >> 16 & 0x000000FF)
         +CharToStr(curr >> 24 & 0x000000FF);
      }
   return (text);
   }  
//+------------------------------------------------------------------+

Here, after FindFirstFileA() and FindNextFileA() function calls, the array of win32_DATA (structure WIN32_FIND_DATA) is changed to an "empty" status, namely, all elements of the array are filled with zeros:

   ArrayInitialize(win32_DATA,0);

If we don't do this, it may happen that the "snatches" of the file name from the preceding call penetrate the process and we get some gobbledygook.


Example Realization: Creating Source Codes Backups

A simple example that demonstrates practical features of the above is the creation of source codes backups in a special directory. For this, we should combine the functions considered in this present article with those in article File Operations via WinAPI. We will then obtain simple script backup.mq4 given below. You can see the full code in the attached file. Here is only its start() function that uses all functions described in both articles:

//+------------------------------------------------------------------+
//| script program start function                                    |
//+------------------------------------------------------------------+
int start()
  {
//----
   string expert[1000]; // must be enough 
   string EAname="";    // EA name
   int EAcounter = 0;   // EAs counter   
   int win32_DATA[80];  
   
   int handle = FindFirstFileA(TerminalPath() + "\experts\*.mq4",win32_DATA);
   EAname = bufferToString(win32_DATA);
   expert[0] = EAname;
   ArrayInitialize(win32_DATA,0);
   
   int i=1;
   while (FindNextFileA(handle,win32_DATA))
      {
      EAname = bufferToString(win32_DATA);
      expert[i] = EAname;
      ArrayInitialize(win32_DATA,0);
      i++;
      if (i>=1000) ArrayResize(expert,2000); // now it will surely be enough
      }
 
   ArrayResize(expert, i);
   int size = i;
   
   if (handle>0) FindClose(handle);
 
   for (i = 0 ; i < size; i++) 
      {
      Print(i,":  ",expert[i]);   
      string backupPathName = backup_folder + "experts\\" + expert[i];
      string originalName = TerminalPath() + "\\experts\\" + expert[i];
      string buffer=ReadFile(originalName);
      WriteFile(backupPathName,buffer);   
      }
   if (size > 0 ) Print("There are ",size," files were copied to folder "+backup_folder+"experts\\");   
//----
   return(0);
  }
//+------------------------------------------------------------------+


Conclusion

It was shown how to perform operations with a group of files of the same type. You can read and process files using the above functions and realizing your own logic. You can make timely backups using an Expert Advisor or synchronize two sets of files in different folders, import quotes from some databases, and so on. It is recommended to control the use of DLLs and not to enable connecting libraries for third-parties' files having the extension of ex4. You must make sure that the program you are launching won't damage the data on your PC.

Translated from Russian by MetaQuotes Software Corp.
Original article: https://www.mql5.com/ru/articles/1543

Attached files |
backup.mq4 (7.7 KB)
CheckFindFile.mq4 (1.95 KB)
Step on New Rails: Custom Indicators in MQL5 Step on New Rails: Custom Indicators in MQL5

I will not list all of the new possibilities and features of the new terminal and language. They are numerous, and some novelties are worth the discussion in a separate article. Also there is no code here, written with object-oriented programming, it is a too serous topic to be simply mentioned in a context as additional advantages for developers. In this article we will consider the indicators, their structure, drawing, types and their programming details, as compared to MQL4. I hope that this article will be useful both for beginners and experienced developers, maybe some of them will find something new.

Here Comes the New MetaTrader 5 and MQL5 Here Comes the New MetaTrader 5 and MQL5

This is just a brief review of MetaTrader 5. I can't describe all the system's new features for such a short time period - the testing started on 2009.09.09. This is a symbolical date, and I am sure it will be a lucky number. A few days have passed since I got the beta version of the MetaTrader 5 terminal and MQL5. I haven't managed to try all its features, but I am already impressed.

Using text files for storing input parameters of Expert Advisors, indicators and scripts Using text files for storing input parameters of Expert Advisors, indicators and scripts

The article describes the application of text files for storing dynamic objects, arrays and other variables used as properties of Expert Advisors, indicators and scripts. The files serve as a convenient addition to the functionality of standard tools offered by MQL languages.

How to create an indicator of non-standard charts for MetaTrader Market How to create an indicator of non-standard charts for MetaTrader Market

Through offline charts, programming in MQL4, and reasonable willingness, you can get a variety of chart types: "Point & Figure", "Renko", "Kagi", "Range bars", equivolume charts, etc. In this article, we will show how this can be achieved without using DLL, and therefore such "two-for-one" indicators can be published and purchased from the Market.