Using WinInet.dll for Data Exchange between Terminals via the Internet

--- | 17 June, 2010

MetaTrader 5 opens up unique opportunities for users, using a number of new user interface elements in its arsenal. Because of this, the functions that were previously unavailable can now be used to the maximum.

In this lesson we will learn to:

The MQL5 CodeBase contains an example of script, which works with the wininet.dll library and shows an example of the server page request. But today we'll go much further, and make the server, not only give us the page, but also to send and store this data for subsequent transfers to other requesting terminals.

Note: for those who don't have access to a server, configured with PHP, we suggest downloading the Denwer kit, and using it as a working platform. Also, we recommend using the Apache server and PHP on your localhost for testing.

To send any request to the server, we will need the 7 major functions of the library.

InternetAttemptConnect  Attempt to locate an Internet connection and establish it
InternetOpen
Initializes the structure for the work of the WinInet library functions. This function must be activated before activating any other functions of the library.
InternetConnect Opens the resource specified by the address HTTP URL or FTP. Returns the descriptor to an open connection
HttpOpenRequest Creates a descriptor for HTTP requests for setting up a connection
HttpSendRequest Sends a query, using the created descriptor
InternetReadFile Reads data, received from the server after the query has been processed
InternetCloseHandle Releases the transferred descriptor

 
A detailed description of all of the functions and their parameters can be found in the MSDN Help system.

The declaration of functions remained the same as in MQL4, with the exception of the use of Unicode calls and line transfers by the link. 

#import "wininet.dll"
int InternetAttemptConnect(int x);
int InternetOpenW(string &sAgent,int lAccessType,string &sProxyName,string &sProxyBypass,int lFlags);
int InternetConnectW(int hInternet,string &szServerName,int nServerPort,string &lpszUsername,string &lpszPassword,int dwService,int dwFlags,int dwContext);
int HttpOpenRequestW(int hConnect,string &Verb,string &ObjectName,string &Version,string &Referer,string &AcceptTypes,uint dwFlags,int dwContext);
int HttpSendRequestW(int hRequest,string &lpszHeaders,int dwHeadersLength,uchar &lpOptional[],int dwOptionalLength);
int HttpQueryInfoW(int hRequest,int dwInfoLevel,int &lpvBuffer[],int &lpdwBufferLength,int &lpdwIndex);
int InternetReadFile(int hFile,uchar &sBuffer[],int lNumBytesToRead,int &lNumberOfBytesRead);
int InternetCloseHandle(int hInet);
#import

//To make it clear, we will use the constant names from wininet.h.
#define OPEN_TYPE_PRECONFIG     0           // use the configuration by default
#define FLAG_KEEP_CONNECTION    0x00400000  // do not terminate the connection
#define FLAG_PRAGMA_NOCACHE     0x00000100  // no cashing of the page
#define FLAG_RELOAD             0x80000000  // receive the page from the server when accessing it
#define SERVICE_HTTP            3           // the required protocol

A detailed description of the flags is located in the same section MSDN for each of the functions. If you wish to see the declaration of other constants and functions, then you can download the original wininet.h file, located in the attachments to the article.

1. Guides for creating and deleting Internet session

The first thing we should do is create a session and open a connection to the host. A session should preferably be created only once during the program initialization (eg, in a function OnInit). Or it can be done at the very beginning of launching the Expert Advisor, but it is important to make sure that its successful creation was done only once before the closing of the session. And it shouldn't be invoked repeated and without need, with every new iteration of the implementation OnStart or OnTimer. It's important to avoid frequent calls and creation of the structures required for each call.

Therefore, we will use only one global class instance for describing the session and connection descriptiors.

   string            Host;       // host name
   int               Port;       // port
   int               Session;    // session descriptor
   int               Connect;    // connection descriptor

bool MqlNet::Open(string aHost,int aPort)
  {
   if(aHost=="")
     {
      Print("-Host is not specified");
      return(false);
     }
   // checking the DLL resolution in the terminal  
   if(!TerminalInfoInteger(TERMINAL_DLLS_ALLOWED))
     {
      Print("-DLL is not allowed");
      return(false);
     }
   // if the session was identifies, then we close
   if(Session>0 || Connect>0) Close();
   // record of attempting to open into the journal
   Print("+Open Inet...");
   // if we were not able to check for the presence of an Internet connection, then we exit
   if(InternetAttemptConnect(0)!=0)
     {
      Print("-Err AttemptConnect");
      return(false);
     }
   string UserAgent="Mozilla"; string nill="";
   // open a session
   Session=InternetOpenW(UserAgent,OPEN_TYPE_PRECONFIG,nill,nill,0);
   // if we were not able to open a session, then exit
   if(Session<=0)
     {
      Print("-Err create Session");
      Close();
      return(false);
     }
   Connect=InternetConnectW(Session,aHost,aPort,nill,nill,SERVICE_HTTP,0,0);
   if(Connect<=0)
     {
      Print("-Err create Connect");
      Close();
      return(false);
     }
   Host=aHost; Port=aPort;
   // otherwise all attempts were successful
   return(true);
  }

After initialization the descriptors Session and Connect can be used in all of the following functions. Once all work is completed and MQL-programs are uninstalled, they must be removed. This is done by using the InternetCloseHandle function.

void MqlNet::CloseInet()
  {
   Print("-Close Inet...");
   if(Session>0) InternetCloseHandle(Session); Session=-1;
   if(Connect>0) InternetCloseHandle(Connect); Connect=-1;
  }

Attention! When working with Internet functions, it is necessary to free up all of the descriptors, derived from them, using InternetCloseHandle.

2. Sending a request to the server and receiving the page

For sending a request and receiving a page in response to this request, we will need the remaining three functions HttpOpenRequest, HttpSendRequest и InternetReadFile. The essence of the receiving the page in response to a request is basically the simple process of saving its contents into a local file.


For the convenience of working with requests and contents we will create two universal functions.

Sending a request:

bool MqlNet::Request(string Verb,string Object,string &Out,bool toFile=false,string addData="",bool fromFile=false)
  {
   if(toFile && Out=="")
     {
      Print("-File is not specified ");
      return(false);
     }
   uchar data[];
   int hRequest,hSend,h;
   string Vers="HTTP/1.1";
   string nill="";
   if(fromFile)
     {
      if(FileToArray(addData,data)<0)
        {
         Print("-Err reading file "+addData);

         return(false);
        }
     } // read file in the array
   else StringToCharArray(addData,data);

   if(Session<=0 || Connect<=0)
     {
      Close();
      if(!Open(Host,Port))
        {
         Print("-Err Connect");
         Close();
         return(false);
        }
     }
   // create a request descriptor
   hRequest=HttpOpenRequestW(Connect,Verb,Object,Vers,nill,nill,FLAG_KEEP_CONNECTION|FLAG_RELOAD|FLAG_PRAGMA_NOCACHE,0);
   if(hRequest<=0)
     {
      Print("-Err OpenRequest");
      InternetCloseHandle(Connect);
      return(false);
     }
   // send request
   // headline for request
   string head="Content-Type: application/x-www-form-urlencoded";
   // sent file
   hSend=HttpSendRequestW(hRequest,head,StringLen(head),data,ArraySize(data)-1);
   if(hSend<=0)
     {
      Print("-Err SendRequest");
      InternetCloseHandle(hRequest);
      Close();
     }
   // read the page 
   ReadPage(hRequest,Out,toFile);
   // close all handles
   InternetCloseHandle(hRequest); 
   InternetCloseHandle(hSend);
   return(true);
  }

Function parameters of MqlNet:: Request:

Reading the contents of the received descriptor

void MqlNet::ReadPage(int hRequest,string &Out,bool toFile)
  {
   // read the page 
   uchar ch[100];
   string toStr="";
   int dwBytes,h;
   while(InternetReadFile(hRequest,ch,100,dwBytes))
     {
      if(dwBytes<=0) break;
      toStr=toStr+CharArrayToString(ch,0,dwBytes);
     }
   if(toFile)
     {
      h=FileOpen(Out,FILE_BIN|FILE_WRITE);
      FileWriteString(h,toStr);
      FileClose(h);
     }
   else Out=toStr;
  }

Function parameters of MqlNet:: ReadPage:

And by gathering all of this into one, we will obtain an MqlNet library class for working with the Internet.

class MqlNet
  {
   string            Host;     // host name
   int               Port;     // port
   int               Session; // session descriptor
   int               Connect; // connection descriptor
public:
                     MqlNet(); // class constructor
                    ~MqlNet(); // destructor
   bool              Open(string aHost,int aPort); // create a session and open a connection
   void              Close(); // close session and connection
   bool              Request(string Verb,string Request,string &Out,bool toFile=false,string addData="",bool fromFile=false); // send request
   bool              OpenURL(string URL,string &Out,bool toFile); // somply read the page into the file or the variable
   void              ReadPage(int hRequest,string &Out,bool toFile); // read the page
   int               FileToArray(string FileName,uchar &data[]); // copy the file into the array for sending
  };

That's basically all of the required functions that are likely to satisfy the diversified needs for working with the Internet. Consider the examples of their use.

Example 1. Automatic download of MQL-programs into the terminal's folders. MetaGrabber script

To begin our testing of the class's work, let us start with the easiest tasks - reading the page and saving its contents to the specified folder. But a simple reading of the pages is unlikely to be very interesting, so in order to gain something from the work of the script, let's assign it a functional of grabber of mql programs from sites. The task of the MetaGrabber script will be:

To solve the second problem we use the MqlNet class. For the third task we use the function MoveFileEx from Kernel32.dll

#import "Kernel32.dll"
bool MoveFileExW(string &lpExistingFileName, string &lpNewFileName, int dwFlags);
#import "Kernel32.dll"

For the first problem, let's make a small service function of parsing the URL line.

We need to allocate three lines from the address: the host, the path to the site, and the file name.
For example, in line http://www.mysite.com/folder/page.html

- Host = www.mysite.com
- Request = / folder / page.html
- File name = page.html

In the case of CodeBase on the MQL5 site, the path ways have the same structure. For example, the path to the library ErrorDescription.mq5 on page https://www.mql5.com/ru/code/79 looks like http://p.mql5.com/data/18/79/ErrorDescription.mqh. This path is easily obtained by right clicking on the link and selecting "Copy Link". Thus, the URL is split into two parts, one for the request and one for the file name for convenience of file storage.

- Host = p.mql5.com
- Request = / data/18/79/5/ErrorDescription.mqh
- File name = ErrorDescription.mqh

This is the kind of line parsing that the following ParseURL function will be dealing with.

void ParseURL(string path,string &host,string &request,string &filename)
  {
   host=StringSubstr(URL,7);
   // removed
   int i=StringFind(host,"/"); 
   request=StringSubstr(host,i);
   host=StringSubstr(host,0,i);
   string file="";
   for(i=StringLen(URL)-1; i>=0; i--)
      if(StringSubstr(URL,i,1)=="/")
        {
         file=StringSubstr(URL,i+1);
         break;
        }
   if(file!="") filename=file;
  }

In the outer parameters of the script we will make only two parameters - URL (path of the mql5 file) and the type of folder of subsequent placement - that is, into which terminal folder you wish to place it.

As a result, we obtain a short but very useful script.

//+------------------------------------------------------------------+
//|                                                  MetaGrabber.mq5 |
//|                                 Copyright © 2010 www.fxmaster.de |
//|                                         Coding by Sergeev Alexey |
//+------------------------------------------------------------------+
#property copyright "www.fxmaster.de  © 2010"
#property link      "www.fxmaster.de"
#property version               "1.00"
#property description  "Download files from internet"

#property script_show_inputs

#include <InternetLib.mqh>

#import "Kernel32.dll"
bool MoveFileExW(string &lpExistingFileName,string &lpNewFileName,int dwFlags);
#import
#define MOVEFILE_REPLACE_EXISTING 0x1

enum _FolderType
  {
   Experts=0,
   Indicators=1,
   Scripts=2,
   Include=3,
   Libraries=4,
   Files=5,
   Templates=6,
   TesterSet=7
  };

input string URL="";
input _FolderType FolderType=0;
//------------------------------------------------------------------ OnStart
int OnStart()
  {
   MqlNet INet; // variable for working in the Internet
   string Host,Request,FileName="Recieve_"+TimeToString(TimeCurrent())+".mq5";

   // parse url
   ParseURL(URL,Host,Request,FileName);

   // open session
   if(!INet.Open(Host,80)) return(0);
   Print("+Copy "+FileName+" from  http://"+Host+" to "+GetFolder(FolderType));

   // obtained file
   if(!INet.Request("GET",Request,FileName,true))
     {
      Print("-Err download "+URL);
      return(0);
     }
   Print("+Ok download "+FileName);

   // move to the target folder
   string to,from,dir;
   // if there is no need to move it elsewhere
   if(FolderType==Files) return(0);

   // from
   from=TerminalInfoString(TERMINAL_DATA_PATH)+"\\MQL5\\Files\\"+FileName;

   // to
   to=TerminalInfoString(TERMINAL_DATA_PATH)+"\\";
   if(FolderType!=Templates && FolderType!=TesterSet) to+="MQL5\\";
   to+=GetFolder(FolderType)+"\\"+FileName;

   // move file 
   if(!MoveFileExW(from,to,MOVEFILE_REPLACE_EXISTING))
     {
      Print("-Err move to "+to);
      return(0);
     }
   Print("+Ok move "+FileName+" to "+GetFolder(FolderType));

   return(0);
  }
//------------------------------------------------------------------ GetFolder
string GetFolder(_FolderType foldertype)
  {
   if(foldertype==Experts) return("Experts");
   if(foldertype==Indicators) return("Indicators");
   if(foldertype==Scripts) return("Scripts");
   if(foldertype==Include) return("Include");
   if(foldertype==Libraries) return("Libraries");
   if(foldertype==Files) return("Files");
   if(foldertype==Templates) return("Profiles\\Templates");
   if(foldertype==TesterSet) return("Tester");
   return("");
  }
//------------------------------------------------------------------ ParseURL
void ParseURL(string path,string &host,string &request,string &filename)
  {
   host=StringSubstr(URL,7);
   // removed
   int i=StringFind(host,"/"); 
   request=StringSubstr(host,i);
   host=StringSubstr(host,0,i);
   string file="";
   for(i=StringLen(URL)-1; i>=0; i--)
      if(StringSubstr(URL,i,1)=="/")
        {
         file=StringSubstr(URL,i+1);
         break;
        }
   if(file!="") filename=file;
  }
//+------------------------------------------------------------------+


Let's conduct the experiments on our favorite section https://www.mql5.com/en/code. The downloaded files will immediately appear in the editor's navigator, and they can be compiled without restarting the terminal or the editor. And they will not wander through the long paths of the file system in the search for the desired folder to move the files into.

Attention! Many sites put up a security against massive content downloading, and your IP-address, in case of such mass download, may be blocked by this resource. So pay careful attention to the use of "machine" downloading of files from resources, which you access often and do not want to be banned from using.

Those who wish to go even further, improving the proposed service, can use the Clipboard script with the interception of the clipboard's contents and the further automatic downloading.

Example 2. Monitoring quotes from multiple brokers on a single chart

So we have learned to obtain files from the Internet. Now let's consider a more interesting question - how to send and store this data on the server. For this we need a small additional PHP-script, which will be located on the server. Using the written MqlNet class, we create an Expert Advisor for monitoring - MetaArbitrage . The task of the expert in conjunction with the PHP-script will be:

The schematic diagram of the interaction between the MQL-module and the PHP-script is as follows:


We will use the MqlNet class to solve these tasks.

To avoid the duplication of data, as well as to weed out outdated quotes - we will send 4 main parameters: the name of the broker's server (the source of current prices), the currency, the price and time of quotes in UTC. For example, a request to access the script from the resources of our company is as follows:

www.fxmaster.de/metaarbitr.php?server=Metaquotes&pair=EURUSD&bid=1.4512&time=13286794

These parameters and the actual quote are stored on the server and will be issued on the response page, along with all other stored quotes of this currency.

The "collateral" convenience of this exchange is that quotes can be sent from MT5, as well as from MT4!

The page, which is formed by the server, is a regular CSV file. In this script it looks like this:

ServerName1; Bid1; Time1
ServerName 2; Bid2; Time2
ServerName 3; Bid3; Time3
…
ServerName N; BidN; TimeN

But you can add to it your own additional parameters (eg, server type - demo or real). We store this CSV-file and parse it line by line, with the output of a table of value and line prices displayed on the screen.

Processing this file can be in be accomplished in many different ways, choosing the once that are required in each particular case. For example, filter out the quotes, received from the MetaTrader 4 demo server, etc.


The advantages of using the Internet server are obvious - you are sending your quotes, which can be received and viewed by another trader. Likewise, you will receive quotes that are sent to other traders. That is, the interaction between the terminals is bilateral, the exchange of data is accomplished as shown in the following scheme:


This scheme serve as the basis for the principle of information exchange between any number of terminals. A full MetaArbitrage Expert Advisor and the PHP-script with commentaries can be downloaded from the link in the attachments. More about PHP-used functions can be read on the following site php.su

Example 3. Exchange of messages (mini chat) within the terminal. MetaChat Expert Advisor

Let's take a step away from trading and numbers, and create an application, which will allow us to chat with several people at once, without exiting the terminal. To do this we will need one more PHP script, which in general is very similar to the previous one. Except for the fact that in this script, instead of analyzing time quotations, we will be analyzing the number of lines in a file. The task of the Expert Advisor will be:

The work of MetaChat will not differ from that of the previous Expert Advisor. The same principle, and the same simple CSV file for output.


MetaChat and MetaArbitrage are maintained on the site of its developers. The PHP-scripts for their work are also located there.
Therefore, if you want to test a work or use this service, you can access it through the following link:
MetaСhat - www.fxmaster.de/metachat.php
MetaArbitrage - www.fxmaster.de/metaarbitr.php

Conclusion

And so we have familiarized ourselves with the HTTP-requests. We gained the ability to send and receive data through the Internet, and to organize the working process more comfortably. But any capabilities can always be improves. The following can be considered the new potential directions of their improvements:

In this article we used the GET type of requests. They sufficiently fulfill the task when you need to obtain a file, or send a request, with a small number of parameters, for a server analyses.

In our next lesson, we will take a careful look at POST requests - sending files to the server or file sharing between terminals, and we will consider the examples of their use.

Useful Resources