Русский 中文 Español Deutsch 日本語 Português 한국어 Français Italiano Türkçe
Communicating With MetaTrader 5 Using Named Pipes Without Using DLLs

Communicating With MetaTrader 5 Using Named Pipes Without Using DLLs

MetaTrader 5Examples | 15 October 2012, 21:16
32 533 66
MetaQuotes
MetaQuotes

Introduction

Many developers face the same problem - how to get to the trading terminal sandbox without using unsafe DLLs.

One of the easiest and safest method is to use standard Named Pipes that work as normal file operations. They allow you to organize interprocessor client-server communication between programs. Although there is an already published article A DLL-free solution to communicate between MetaTrader 5 terminals using Named Pipes on this topic that demonstrates enabling access to DLLs, we will use standard and safe features of client terminal.

You can find more information about named pipes in MSDN library, but we will get down to practical examples in C++ and MQL5. We will implement server, client, data exchange between them and then benchmark performance.


Server Implementation

Let's code a simple server in C++. A script from the terminal will connect to this server and will exchange data with it. The server core has the following set of WinAPI functions:

Once a named pipe is opened it returns a file handle that can be used for regular read/write file operations. As a result you get a very simple mechanism that don't require any special knowledge in network operations.

Named pipes have one distinctive feature - they can be both local and network. That is, it's easy to implement a remote server that will accept network connections from client terminals.

Here is a simple example of creating a local server as full-duplex channel that works in the bytes exchange mode:

//--- open 
CPipeManager manager;

if(!manager.Create(L"\\\\.\\pipe\\MQL5.Pipe.Server"))
   return(-1);


//+------------------------------------------------------------------+
//| Create named pipe                                                |
//+------------------------------------------------------------------+
bool CPipeManager::Create(LPCWSTR pipename)
  {
//--- check parameters
   if(!pipename || *pipename==0) return(false);
//--- close old
   Close();
//--- create named pipe
   m_handle=CreateNamedPipe(pipename,PIPE_ACCESS_DUPLEX,
                            PIPE_TYPE_BYTE | PIPE_READMODE_BYTE | PIPE_WAIT,
                            PIPE_UNLIMITED_INSTANCES,256*1024,256*1024,1000,NULL);

   if(m_handle==INVALID_HANDLE_VALUE)
     {
      wprintf(L"Creating pipe '%s' failed\n",pipename);
      return(false);
     }
//--- ok
   wprintf(L"Pipe '%s' created\n",pipename);
   return(true);
  }

To get a client connection you have to use the ConnectNamedPipe function:

//+------------------------------------------------------------------+
//| Connect client                                                   |
//+------------------------------------------------------------------+
bool CPipeManager::ConnectClient(void)
  {
//--- pipe exists?
   if(m_handle==INVALID_HANDLE_VALUE) return(false);
//--- connected?
   if(!m_connected)
     {
      //--- connect
      if(ConnectNamedPipe(m_handle,NULL)==0)
        {
         //--- client already connected before ConnectNamedPipe?
         if(GetLastError()!=ERROR_PIPE_CONNECTED)
            return(false);
         //--- ok
        }
      m_connected=true;
     }
//---
   return(true);
  }

Data exchange is organized using 4 simple functions:

  • CPipeManager::Send(void *data,size_t data_size)
  • CPipeManager::Read(void *data,size_t data_size)
  • CPipeManager::SendString(LPCSTR command)
  • CPipeManager::ReadString(LPSTR answer,size_t answer_maxlen)

They allow you to send/receive data as binary data or ANSI text strings in MQL5 compatible mode. Moreover, since CFilePipe in MQL5 opens a file in ANSI mode by default, strings are automatically converted to Unicode on receipt and sending. If your MQL5 program opens a file in Unicode mode (FILE_UNICODE), then it can exchange Unicode strings (with BOM starting signature).


Client Implementation

We will write our client in MQL5. It will be able to perform regular file operations using the CFilePipe class from Standard Library. This class is almost identical to the CFileBin, but it contains an important verification of data availability in a virtual file before reading this data.

//+------------------------------------------------------------------+
//| Wait for incoming data                                           |
//+------------------------------------------------------------------+
bool CFilePipe::WaitForRead(const ulong size)
  {
//--- check handle and stop flag
   while(m_handle!=INVALID_HANDLE && !IsStopped())
     {
      //--- enough data?
      if(FileSize(m_handle)>=size)
         return(true);
      //--- wait a little
      Sleep(1);
     }
//--- failure
   return(false);
  }

//+------------------------------------------------------------------+
//| Read an array of variables of double type                        |
//+------------------------------------------------------------------+
uint CFilePipe::ReadDoubleArray(double &array[],const int start_item,const int items_count)
  {
//--- calculate size
   uint size=ArraySize(array);
   if(items_count!=WHOLE_ARRAY) size=items_count;
//--- check for data
   if(WaitForRead(size*sizeof(double)))
      return FileReadArray(m_handle,array,start_item,items_count);
//--- failure
   return(0);
  }

Named pipes have significant differences in implementation of their local and network modes. Without such a verification, network mode operations will always return a read error when sending large amounts of data (over 64K).

Let's connect to the server with two checks: either to remote computer named 'RemoteServerName' or to local machine.

void OnStart()
  {
//--- wait for pipe server
   while(!IsStopped())
     {
      if(ExtPipe.Open("\\\\RemoteServerName\\pipe\\MQL5.Pipe.Server",FILE_READ|FILE_WRITE|FILE_BIN)!=INVALID_HANDLE) break;
      if(ExtPipe.Open("\\\\.\\pipe\\MQL5.Pipe.Server",FILE_READ|FILE_WRITE|FILE_BIN)!=INVALID_HANDLE) break;
      Sleep(250);
     }
   Print("Client: pipe opened");


Data Exchange

After successful connection let's send a text string with identification info to the server. Unicode string will be automatically converted into ANSI, since the file is opened in ANSI mode.

//--- send welcome message
   if(!ExtPipe.WriteString(__FILE__+" on MQL5 build "+IntegerToString(__MQ5BUILD__)))
     {
      Print("Client: sending welcome message failed");
      return;
     }

In response, the server will send its string "Hello from pipe server" and the integer 1234567890. The client once again will send string "Test string" and the integer 1234567890.

//--- read data from server
   string        str;
   int           value=0;

   if(!ExtPipe.ReadString(str))
     {
      Print("Client: reading string failed");
      return;
     }
   Print("Server: ",str," received");

   if(!ExtPipe.ReadInteger(value))
     {
      Print("Client: reading integer failed");
      return;
     }
   Print("Server: ",value," received");
//--- send data to server
   if(!ExtPipe.WriteString("Test string"))
     {
      Print("Client: sending string failed");
      return;
     }

   if(!ExtPipe.WriteInteger(value))
     {
      Print("Client: sending integer failed");
      return;
     }

OK, we are finished with simple data exchange. Now it's time for performance benchmark.


Performance Benchmark

As a test, we will send 1 gigabyte of data as an array of the double type numbers in blocks of 8 megabytes from server to client, then check correctness of the blocks and measure the transfer rate.

Here is this code in C++ server:

//--- benchmark
   double  volume=0.0;
   double *buffer=new double[1024*1024];   // 8 Mb

   wprintf(L"Server: start benchmark\n");
   if(buffer)
     {
      //--- fill the buffer
      for(size_t j=0;j<1024*1024;j++)
         buffer[j]=j;
      //--- send 8 Mb * 128 = 1024 Mb to client
      DWORD   ticks=GetTickCount();

      for(size_t i=0;i<128;i++)
        {
         //--- setup guard signatures
         buffer[0]=i;
         buffer[1024*1024-1]=i+1024*1024-1;
         //--- 
         if(!manager.Send(buffer,sizeof(double)*1024*1024))
           {
            wprintf(L"Server: benchmark failed, %d\n",GetLastError());
            break;
           }
         volume+=sizeof(double)*1024*1024;
         wprintf(L".");
        }
      wprintf(L"\n");
      //--- read confirmation
      if(!manager.Read(&value,sizeof(value)) || value!=12345)
         wprintf(L"Server: benchmark confirmation failed\n");
      //--- show statistics
      ticks=GetTickCount()-ticks;
      if(ticks>0)
         wprintf(L"Server: %.0lf Mb sent at %.0lf Mb per second\n",volume/1024/1024,volume/1024/ticks);
      //---
      delete[] buffer;
     }

and in MQL5 client:

//--- benchmark
   double buffer[];
   double volume=0.0;

   if(ArrayResize(buffer,1024*1024,0)==1024*1024)
     {
      uint  ticks=GetTickCount();
      //--- read 8 Mb * 128 = 1024 Mb from server
      for(int i=0;i<128;i++)
        {
         uint items=ExtPipe.ReadDoubleArray(buffer);
         if(items!=1024*1024)
           {
            Print("Client: benchmark failed after ",volume/1024," Kb, ",items," items received");
            break;
           }
         //--- check the data
         if(buffer[0]!=i || buffer[1024*1024-1]!=i+1024*1024-1)
           {
            Print("Client: benchmark invalid content");
            break;
           }
         //---
         volume+=sizeof(double)*1024*1024;
        }
      //--- send confirmation
      value=12345;
      if(!ExtPipe.WriteInteger(value))
         Print("Client: benchmark confirmation failed ");
      //--- show statistics
      ticks=GetTickCount()-ticks;
      if(ticks>0)
         printf("Client: %.0lf Mb received at %.0lf Mb per second\n",volume/1024/1024,volume/1024/ticks);
      //---
      ArrayFree(buffer);
     }

Note, that the first and the last elements of transfered blocks are checked in order to make sure that there were no errors during the transfer. Also, when transfer is complete client sends a confirming signal to the server about successful data receipt. If you won't use final confirmations, you will easily encounter a data loss if one of the parties closes connection too early.

Run the PipeServer.exe server locally and attach the PipeClient.mq5 script to any chart:

PipeServer.exe PipeClient.mq5
MQL5 Pipe Server
Copyright 2012, MetaQuotes Software Corp.
Pipe '\\.\pipe\MQL5.Pipe.Server' created
Client: waiting for connection...
Client: connected as 'PipeClient.mq5 on MQL5 build 705'
Server: send string
Server: send integer
Server: read string
Server: 'Test string' received
Server: read integer
Server: 1234567890 received
Server: start benchmark
......................................................
........
Server: 1024 Mb sent at 2921 Mb per second
PipeClient (EURUSD,H1)  Client: pipe opened
PipeClient (EURUSD,H1)  Server: Hello from pipe server received
PipeClient (EURUSD,H1)  Server: 1234567890 received
PipeClient (EURUSD,H1)  Client: 1024 Mb received at 2921 Mb per second


For local exchange, transfer rate is truly amazing - almost 3 gigabytes per second. This means that named pipes can be used to transfer almost any amount of data into MQL5 programs.

Now let's benchmark data transfer performance in an ordinary 1 gigabit LAN:

PipeServer.exe PipeClient.mq5
MQL5 Pipe Server
Copyright 2012, MetaQuotes Software Corp.
Pipe '\\.\pipe\MQL5.Pipe.Server' created
Client: waiting for connection...
Client: connected as 'PipeClient.mq5 on MQL5 build 705'
Server: send string
Server: send integer
Server: read string
Server: 'Test string' received
Server: read integer
Server: 1234567890 received
Server: start benchmark
......................................................
........
Server: 1024 Mb sent at 63 Mb per second
PipeClient (EURUSD,H1)  Client: pipe opened
PipeClient (EURUSD,H1)  Server: Hello from pipe server received
PipeClient (EURUSD,H1)  Server: 1234567890 received
PipeClient (EURUSD,H1)  Client: 1024 Mb received at 63 Mb per second


In local network, 1 gigabyte of data has been transfered at rate of 63 megabytes per second, which is very good. In fact it is 63% of the gigabit network maximum bandwidth.


Conclusion

Protection system of the MetaTrader 5 trading platform does not allow MQL5 programs run outside their sandbox, guarding traders against threats when using untrusted Expert Advisors. Using named pipes you can easy create integrations with third-party software and manage EAs from outside. Safely.

Translated from Russian by MetaQuotes Ltd.
Original article: https://www.mql5.com/ru/articles/503

Attached files |
pipeclient.mq5 (3.15 KB)
pipeserver.zip (43.59 KB)
Last comments | Go to discussion (66)
[Deleted] | 14 Feb 2018 at 08:30
Or are the pips no longer relevant?
leonerd
leonerd | 11 Sep 2020 at 19:03

Is the server for one client? I'm trying to connect a second client, the connection doesn't open. 5004 error. The file name is the same as in the other client connected.

If I disconnect the first client, the second one connects. So one named channel is only one connection?

leonerd
leonerd | 11 Sep 2020 at 21:05

How to connect two MT clients via named channels?

Tried the code from here https://www.mql5.com/en/articles/115. It doesn't work. Connect method hangs.

Pahlavon Tursunaliyev
Pahlavon Tursunaliyev | 11 Jul 2022 at 00:59
Salom
Quantum Capital International Group Ltd
Yang Chih Chou | 30 Aug 2023 at 09:00
Can it possible to use in C#?
Interview with Francisco García García (ATC 2012) Interview with Francisco García García (ATC 2012)
Today we interview Francisco García García (chuliweb) from Spain. A week ago his Expert Advisor reached the 8th place, but the unfortunate logic error in programming threw it from the first page of the Championship leaders. As confirmed by statistics, such an error is not uncommon for many participants.
Statistical Carry Trade Strategy Statistical Carry Trade Strategy
An algorithm of statistical protection of open positive swap positions from unwanted price movements. This article features a variant of the carry trade protection strategy that allows to compensate for potential risk of the price movement in the direction opposite to that of the open position.
How to Write a Good Description for a Market Product How to Write a Good Description for a Market Product
MQL5 Market has many products for sale but some of their descriptions leave much to be desired. Many texts are obviously in need of improvement, as common traders are not able to comprehend them. This article will help you to put your product in a favorable light. Use our recommendations to write an eye-catching description that will easily show your customers what exactly you are selling.
How to Subscribe to Trading Signals How to Subscribe to Trading Signals
The Signals service introduces social trading with MetaTrader 4 and MetaTrader 5. The Service is integrated into the trading platform, and allows anyone to easily copy trades of professional traders. Select any of the thousands of signal providers, subscribe in a few clicks and the provider's trades will be copied on your account.