Send orders to MT4 from Java via IP - page 3

JC 2016.09.17 08:26 #21

jjc:

... stating the obvious, if you wanted to do the above in an EA rather than a script [...]

Revised version which can be used to create an EA rather than a script simply by un-commenting the #define COMPILE_AS_EA. The code then has OnTimer (and OnTick) instead of OnStart.

The only other change is that the code now puts the sockets into non-blocking mode. It still polls for the availability of new socket events using select(), but it now avoids the danger of entering into a blocking state if select() is somehow erroneous.

#property strict 

// ---------------------------------------------------------------------
// This code can either create an EA or a script. To create an EA,
// remove the commenting-out of the line below. The code will then
// have OnTimer() and OnTick() instead of OnStart()
// ---------------------------------------------------------------------

// #define COMPILE_AS_EA


// ---------------------------------------------------------------------
// If we are compiling as a script, then we want the script
// to display the inputs page
// ---------------------------------------------------------------------

#ifndef COMPILE_AS_EA
#property show_inputs
#endif


// ---------------------------------------------------------------------
// User-configurable parameters
// ---------------------------------------------------------------------

input int PortNumber = 51234; // TCP/IP port number

// ---------------------------------------------------------------------
// Constants
// ---------------------------------------------------------------------

// Size of temporary buffer used to read from client sockets
#define SOCKET_READ_BUFFER_SIZE  10000

// Defines the frequency of the main-loop processing, in milliseconds.
// This is either the duration of the sleep in OnStart(), if compiled
// as a script, or the frequency which is given to EventSetMillisecondTimer()
// if compiled as an EA
#define SLEEP_MILLISECONDS       10


// ---------------------------------------------------------------------
// Forward definitions of classes
// ---------------------------------------------------------------------

// Wrapper around a connected client socket
class Connection;

// ---------------------------------------------------------------------
// Winsock structure definitions and DLL imports
// ---------------------------------------------------------------------

struct sockaddr_in {
   short af_family;
   short port;
   int addr;
   int dummy1;
   int dummy2;
};

struct timeval {
   int secs;
   int usecs;
};

struct fd_set {
   int count;
   int single_socket;
   int dummy[63];
};

#import "Ws2_32.dll"
   int socket(int, int, int);   
   int bind(int, sockaddr_in&, int);
   int htons(int);
   int listen(int, int);
   int accept(int, int, int);
   int closesocket(int);
   int select(int, fd_set&, int, int, timeval&);
   int recv(int, uchar&[], int, int);
   int ioctlsocket(int, uint, uint&);
   int WSAGetLastError();
#import   


// ---------------------------------------------------------------------
// Global variables
// ---------------------------------------------------------------------

// Flags whether OnInit was successful
bool SuccessfulInit = false;

// Handle of main listening server socket
int ServerSocket;

// List of currently connected clients
Connection * Clients[];

// For EA compilation only, we track whether we have 
// successfully created a timer
#ifdef COMPILE_AS_EA
bool CreatedTimer = false;   
#endif

// ---------------------------------------------------------------------
// Initialisation - create listening socket
// ---------------------------------------------------------------------

void OnInit()
{
   SuccessfulInit = false;
   
   if (!TerminalInfoInteger(TERMINAL_DLLS_ALLOWED)) {Print("Requires \'Allow DLL imports\'");return;}

   // (Don't need to call WSAStartup because MT4 must have done this)

   // Create the main server socket
   ServerSocket = socket(2 /* AF_INET */, 1 /* SOCK_STREAM */, 6 /* IPPROTO_TCP */);
   if (ServerSocket == -1) {Print("ERROR " , WSAGetLastError() , " in socket creation");return;}
   
   // Put the socket into non-blocking mode
   uint nbmode = 1;
   if (ioctlsocket(ServerSocket, 0x8004667E /* FIONBIO */, nbmode) != 0) {Print("ERROR in setting non-blocking mode on server socket");return;}
   
   // Bind the socket to the specified port number. In this example,
   // we only accept connections from localhost
   sockaddr_in service;
   service.af_family = 2 /* AF_INET */;
   service.addr = 0x100007F; // equivalent to inet_addr("127.0.0.1")
   service.port = (short)htons(PortNumber);
   if (bind(ServerSocket, service, 16 /* sizeof(service) */) == -1) {Print("ERROR " , WSAGetLastError() , " in socket bind");return;}

   // Put the socket into listening mode
   if (listen(ServerSocket, 10) == -1) {Print("ERROR " , WSAGetLastError() , " in socket listen");return;}

   // Flag that we've successfully initialised
   SuccessfulInit = true;

   // If we're operating as an EA rather than a script, then try setting up a timer
   #ifdef COMPILE_AS_EA
   CreateTimer();
   #endif   
}


// ---------------------------------------------------------------------
// Variation between EA and script, depending on COMPILE_AS_EA above.
// Both versions simply call MainLoop() to handle the socket activity.
// The script version simply does so from a loop in OnStart().
// The EA version sets up a timer, and then calls MainLoop from OnTimer()
// ---------------------------------------------------------------------

#ifdef COMPILE_AS_EA
   // .........................................................
   // EA version. Simply calls MainLoop() from a timer.
   // The timer has previously been created in OnInit(), 
   // though we also check this in OnTick()...
   void OnTimer()
   {
      MainLoop();
   }
   
   // The function which creates the timer if it hasn't yet been set up
   void CreateTimer()
   {
      if (!CreatedTimer) CreatedTimer = EventSetMillisecondTimer(SLEEP_MILLISECONDS);
   }

   // Timer creation can sometimes fail in OnInit(), at least
   // up until MT4 build 970, if MT4 is starting up with the EA already
   // attached to a chart. Therefore, as a fallback, we also 
   // handle OnTick(), and thus eventually set up the timer in
   // the EA's first tick if we failed to set it up in OnInit()
   void OnTick()
   {
      CreateTimer();
   }

#else

   // .........................................................
   // Script version. Simply calls MainLoop() repeatedly
   // until the script is removed, with a small sleep between calls
   void OnStart()
   {
      while (!IsStopped()) {
         MainLoop();
         Sleep(SLEEP_MILLISECONDS);
      }
   }
#endif


// ---------------------------------------------------------------------
// Main processing loop which handles new incoming connections, and reads
// data from existing connections.
// Can either be called from OnStart() in a script, on a continuous loop 
// until IsStopped() is true; or from OnTimer() in an EA.
// ---------------------------------------------------------------------

void MainLoop()
{
   // Do nothing if init was unsuccessful
   if (!SuccessfulInit) return;
   
   // .........................................................
   // Do we have a new pending connection on the server socket?
   timeval waitfor;
   waitfor.secs = 0;
   waitfor.usecs = 0;
   
   fd_set PollServerSocket;
   PollServerSocket.count = 1;
   PollServerSocket.single_socket = ServerSocket;

   int selres = select(0, PollServerSocket, 0, 0, waitfor);
   if (selres > 0) {
   
      Print("New incoming connection...");
      int NewClientSocket = accept(ServerSocket, 0, 0);
      if (NewClientSocket == -1) {
         if (WSAGetLastError() == 10035 /* WSAEWOULDBLOCK */) {
            // Blocking warning; ignore
         } else {
            Print("ERROR " , WSAGetLastError() , " in socket accept");
         }

      } else {
         Print("...accepted");

         int ctarr = ArraySize(Clients);
         ArrayResize(Clients, ctarr + 1);
         Clients[ctarr] = new Connection(NewClientSocket);               
         Print("Got connection to client #", Clients[ctarr].GetID());
      }
   }

   // .........................................................
   // Process any incoming data from client connections
   // (including any which have just been accepted, above)
   int ctarr = ArraySize(Clients);
   for (int i = ctarr - 1; i >= 0; i--) {
      // Return value from ReadAnyPendingData() is true
      // if the socket still seems to be alive; false if 
      // the connection seems to have been closed, and should be discarded
      if (Clients[i].ReadAnyPendingData()) {
         // Socket still seems to be alive
         
      } else {
         // Socket appears to be dead. Delete, and remove from list
         Print("Lost connection to client #", Clients[i].GetID());
         
         delete Clients[i];
         for (int j = i + 1; j < ctarr; j++) {
            Clients[j - 1] = Clients[j];
         }
         ctarr--;
         ArrayResize(Clients, ctarr);           
      }
   }
}

// ---------------------------------------------------------------------
// Termination - clean up sockets
// ---------------------------------------------------------------------

void OnDeinit(const int reason)
{
   closesocket(ServerSocket);
   
   for (int i = 0; i < ArraySize(Clients); i++) {
      delete Clients[i];
   }
   ArrayResize(Clients, 0);
   
   #ifdef COMPILE_AS_EA
   EventKillTimer();
   CreatedTimer = false;
   #endif
}


// ---------------------------------------------------------------------
// Simple wrapper around each connected client socket
// ---------------------------------------------------------------------

class Connection {
private:
   // Client socket handle
   int mSocket;

   // Temporary buffer used to handle incoming data
   uchar mTempBuffer[SOCKET_READ_BUFFER_SIZE];
   
   // Stored-up data, waiting for a \r character 
   string mPendingData;
   
public:
   Connection(int ClientSocket);
   ~Connection();
   string GetID() {return IntegerToString(mSocket);}
   
   bool ReadAnyPendingData();
   void ProcessIncomingMessage(string strMessage);
};

// Constructor, called with the handle of a newly accepted socket
Connection::Connection(int ClientSocket)
{
   mPendingData = "";
   mSocket = ClientSocket; 
   
   // Put the client socket into non-blocking mode
   uint nbmode = 1;
   if (ioctlsocket(mSocket, 0x8004667E /* FIONBIO */, nbmode) != 0) {Print("ERROR in setting non-blocking mode on client socket");}
}

// Destructor. Simply close the client socket
Connection::~Connection()
{
   closesocket(mSocket);
}

// Called repeatedly on a timer, to check whether any
// data is available on this client connection. Returns true if the 
// client still seems to be connected (*not* if there's new data); 
// returns false if the connection seems to be dead. 
bool Connection::ReadAnyPendingData()
{
   // Check the client socket for data-readability
   timeval waitfor;
   waitfor.secs = 0;
   waitfor.usecs = 0;

   fd_set PollClientSocket;
   PollClientSocket.count = 1;
   PollClientSocket.single_socket = mSocket;

   int selres = select(0, PollClientSocket, 0, 0, waitfor);
   if (selres > 0) {
      
      // Winsock says that there is data waiting to be read on this socket
      int res = recv(mSocket, mTempBuffer, SOCKET_READ_BUFFER_SIZE, 0);
      if (res > 0) {
         // Convert the buffer to a string, and add it to any pending
         // data which we already have on this connection
         string strIncoming = CharArrayToString(mTempBuffer, 0, res);
         mPendingData += strIncoming;
         
         // Do we have a complete message (or more than one) ending in \r?
         int idxTerm = StringFind(mPendingData, "\r");
         while (idxTerm >= 0) {
            if (idxTerm > 0) {
               string strMsg = StringSubstr(mPendingData, 0, idxTerm);         
               
               // Strip out any \n characters lingering from \r\n combos
               StringReplace(strMsg, "\n", "");
               
               // Print the \r-terminated message in the log
               ProcessIncomingMessage(strMsg);
            }               
         
            // Keep looping until we have extracted all the \r delimited 
            // messages, and leave any residue in the pending data 
            mPendingData = StringSubstr(mPendingData, idxTerm + 1);
            idxTerm = StringFind(mPendingData, "\r");
         }
         
         return true;
      
      } else if (WSAGetLastError() == 10035 /* WSAEWOULDBLOCK */) {
         // Would block; not an error
         return true;
         
      } else {
         // recv() failed. Assume socket is dead
         return false;
      }
   
   } else if (selres == -1) {
      // Assume socket is dead
      return false;
      
   } else {
      // No pending data
      return true;
   }
}

// Can override this with whatever you want to do with incoming messages:
// handling trading commands, send data back down the socket etc etc etc
void Connection::ProcessIncomingMessage(string strMessage)
{
   Print("#" , GetID() , ": " , strMessage);
}

MQL5 Wizard: Development of MetaEditor - Professional editor MetaTrader 5 Built-in Trading

JC 2016.09.19 14:25 #22

jjc:

Revised version which [...]

I'm sure everyone's eager for an update on this...

In the context of an EA - which is now the default in the code - it turns out to be possible to use WSAAsyncSelect() to do event-driven handling of the socket activity. This can often push latency between message-send and message-receipt below the millisecond level. The exact speed varies depending on what else MT4 is doing, but it's on average a lot faster, and never worse, than just using the timer.

It's not possible to get WSAAsyncSelect() to trigger OnTimer() or OnTick() in the EA. For example, WSAAsyncSelect() can be told to fire WM_TIMER messages, but MT4 ignores these because the wparam timer ID from WSAAsyncSelect() doesn't match the timer ID which it is expecting from EventSetMillisecondTimer().

However, telling WSAAsyncSelect() to send WM_KEYDOWN does successfully fire OnChartEvent() in the EA. See the comments in the code for full details of how this addition works.

#property strict 

// ---------------------------------------------------------------------
// This code can either create an EA or a script. To create a script,
// comment out the line below. The code will then have OnTimer() 
// and OnTick() instead of OnStart()
// ---------------------------------------------------------------------

#define COMPILE_AS_EA


// ---------------------------------------------------------------------
// If we are compiling as a script, then we want the script
// to display the inputs page
// ---------------------------------------------------------------------

#ifndef COMPILE_AS_EA
#property show_inputs
#endif


// ---------------------------------------------------------------------
// User-configurable parameters
// ---------------------------------------------------------------------

input int PortNumber = 51234; // TCP/IP port number

// ---------------------------------------------------------------------
// Constants
// ---------------------------------------------------------------------

// Size of temporary buffer used to read from client sockets
#define SOCKET_READ_BUFFER_SIZE  10000

// Defines the frequency of the main-loop processing, in milliseconds.
// This is either the duration of the sleep in OnStart(), if compiled
// as a script, or the frequency which is given to EventSetMillisecondTimer()
// if compiled as an EA
#define SLEEP_MILLISECONDS       10


// ---------------------------------------------------------------------
// Forward definitions of classes
// ---------------------------------------------------------------------

// Wrapper around a connected client socket
class Connection;

// ---------------------------------------------------------------------
// Winsock structure definitions and DLL imports
// ---------------------------------------------------------------------

struct sockaddr_in {
   short af_family;
   short port;
   int addr;
   int dummy1;
   int dummy2;
};

struct timeval {
   int secs;
   int usecs;
};

struct fd_set {
   int count;
   int single_socket;
   int dummy[63];
};

#import "Ws2_32.dll"
   int socket(int, int, int);   
   int bind(int, sockaddr_in&, int);
   int htons(int);
   int listen(int, int);
   int accept(int, int, int);
   int closesocket(int);
   int select(int, fd_set&, int, int, timeval&);
   int recv(int, uchar&[], int, int);
   int ioctlsocket(int, uint, uint&);
   int WSAGetLastError();
   int WSAAsyncSelect(int, int, uint, int);
#import   


// ---------------------------------------------------------------------
// Global variables
// ---------------------------------------------------------------------

// Flags whether OnInit was successful
bool SuccessfulInit = false;

// Handle of main listening server socket
int ServerSocket;

// List of currently connected clients
Connection * Clients[];

// For EA compilation only, we track whether we have 
// successfully created a timer
#ifdef COMPILE_AS_EA
bool CreatedTimer = false;   
#endif

// ---------------------------------------------------------------------
// Initialisation - create listening socket
// ---------------------------------------------------------------------

void OnInit()
{
   SuccessfulInit = false;
   
   if (!TerminalInfoInteger(TERMINAL_DLLS_ALLOWED)) {Print("Requires \'Allow DLL imports\'");return;}
   
   // (Don't need to call WSAStartup because MT4 must have done this)

   // Create the main server socket
   ServerSocket = socket(2 /* AF_INET */, 1 /* SOCK_STREAM */, 6 /* IPPROTO_TCP */);
   if (ServerSocket == -1) {Print("ERROR " , WSAGetLastError() , " in socket creation");return;}
   
   // Put the socket into non-blocking mode
   uint nbmode = 1;
   if (ioctlsocket(ServerSocket, 0x8004667E /* FIONBIO */, nbmode) != 0) {Print("ERROR in setting non-blocking mode on server socket");return;}
   
   // Bind the socket to the specified port number. In this example,
   // we only accept connections from localhost
   sockaddr_in service;
   service.af_family = 2 /* AF_INET */;
   service.addr = 0x100007F; // equivalent to inet_addr("127.0.0.1")
   service.port = (short)htons(PortNumber);
   if (bind(ServerSocket, service, 16 /* sizeof(service) */) == -1) {Print("ERROR " , WSAGetLastError() , " in socket bind");return;}

   // Put the socket into listening mode
   if (listen(ServerSocket, 10) == -1) {Print("ERROR " , WSAGetLastError() , " in socket listen");return;}

   #ifdef COMPILE_AS_EA
   // In the context of an EA, we can improve the latency even further by making the 
   // processing event-driven, rather than just on a timer. If we use
   // WSAAsyncSelect() to tell Windows to send a WM_KEYDOWN whenever there's socket
   // activity, then this will fire the EA's OnChartEvent. We can thus collect
   // socket events even faster than via the timed check, which becomes a back-up.
   // Note that WSAAsyncSelect() on a server socket also automatically applies 
   // to all client sockets created from it by accept(), and also that
   // WSAAsyncSelect() puts the socket into non-blocking mode, duplicating what
   // we have already done above using ioctlsocket()
   if (WSAAsyncSelect(ServerSocket, (int)ChartGetInteger(0, CHART_WINDOW_HANDLE), 0x100 /* WM_KEYDOWN */, 0xFF /* All events */) != 0) {
      Print("ERROR " , WSAGetLastError() , " in async select");
   }
   #endif

   // Flag that we've successfully initialised
   SuccessfulInit = true;

   // If we're operating as an EA rather than a script, then try setting up a timer
   #ifdef COMPILE_AS_EA
   CreateTimer();
   #endif   
}


// ---------------------------------------------------------------------
// Variation between EA and script, depending on COMPILE_AS_EA above.
// Both versions simply call MainLoop() to handle the socket activity.
// The script version simply does so from a loop in OnStart().
// The EA version sets up a timer, and then calls MainLoop from OnTimer().
// It also has event-driven handling of socket activity by getting
// Windows to simulate keystrokes whenever a socket event happens.
// This is even faster than relying on the timer.
// ---------------------------------------------------------------------

#ifdef COMPILE_AS_EA
   // .........................................................
   // EA version. Simply calls MainLoop() from a timer.
   // The timer has previously been created in OnInit(), 
   // though we also check this in OnTick()...
   void OnTimer()
   {
      MainLoop();
   }
   
   // The function which creates the timer if it hasn't yet been set up
   void CreateTimer()
   {
      if (!CreatedTimer) CreatedTimer = EventSetMillisecondTimer(SLEEP_MILLISECONDS);
   }

   // Timer creation can sometimes fail in OnInit(), at least
   // up until MT4 build 970, if MT4 is starting up with the EA already
   // attached to a chart. Therefore, as a fallback, we also 
   // handle OnTick(), and thus eventually set up the timer in
   // the EA's first tick if we failed to set it up in OnInit()
   void OnTick()
   {
      CreateTimer();
   }

   // In an EA, the use of WSAAsyncSelect() above means that Windows will fire a key-down 
   // message whenever there is socket activity. We can then respond to KEYDOWN events
   // in MQL4 as a way of knowing that there is socket activity to be dealt with.
   void OnChartEvent(const int id, const long& lparam, const double& dparam, const string& sparam)
   {
      if (id == CHARTEVENT_KEYDOWN) {
         // The lparam will be the socket handle, and the dparam will be the type
         // of socket event (e.g. dparam = 1 = FD_READ). The fastest option would be
         // to use the socket handle in lparam to identify the specific socket
         // which requires action, but we'll take the slightly less efficient but simpler
         // route where we just do all our full socket handling in response to an event.
         // Similarly, we won't bother to distinguish between real keypresses and 
         // the pseudo-keypresses from WSAAyncSelect().
         MainLoop();   
      }
   }
  
#else

   // .........................................................
   // Script version. Simply calls MainLoop() repeatedly
   // until the script is removed, with a small sleep between calls
   void OnStart()
   {
      while (!IsStopped()) {
         MainLoop();
         Sleep(SLEEP_MILLISECONDS);
      }
   }
#endif


// ---------------------------------------------------------------------
// Main processing loop which handles new incoming connections, and reads
// data from existing connections.
// Can either be called from OnStart() in a script, on a continuous loop 
// until IsStopped() is true; or from OnTimer() in an EA.
// ---------------------------------------------------------------------

void MainLoop()
{
   // Do nothing if init was unsuccessful
   if (!SuccessfulInit) return;
   
   // .........................................................
   // Do we have a new pending connection on the server socket?
   timeval waitfor;
   waitfor.secs = 0;
   waitfor.usecs = 0;
   
   fd_set PollServerSocket;
   PollServerSocket.count = 1;
   PollServerSocket.single_socket = ServerSocket;

   int selres = select(0, PollServerSocket, 0, 0, waitfor);
   if (selres > 0) {
   
      Print("New incoming connection...");
      int NewClientSocket = accept(ServerSocket, 0, 0);
      if (NewClientSocket == -1) {
         if (WSAGetLastError() == 10035 /* WSAEWOULDBLOCK */) {
            // Blocking warning; ignore
         } else {
            Print("ERROR " , WSAGetLastError() , " in socket accept");
         }

      } else {
         Print("...accepted");

         int ctarr = ArraySize(Clients);
         ArrayResize(Clients, ctarr + 1);
         Clients[ctarr] = new Connection(NewClientSocket);               
         Print("Got connection to client #", Clients[ctarr].GetID());
      }
   }

   // .........................................................
   // Process any incoming data from client connections
   // (including any which have just been accepted, above)
   int ctarr = ArraySize(Clients);
   for (int i = ctarr - 1; i >= 0; i--) {
      // Return value from ReadAnyPendingData() is true
      // if the socket still seems to be alive; false if 
      // the connection seems to have been closed, and should be discarded
      if (Clients[i].ReadAnyPendingData()) {
         // Socket still seems to be alive
         
      } else {
         // Socket appears to be dead. Delete, and remove from list
         Print("Lost connection to client #", Clients[i].GetID());
         
         delete Clients[i];
         for (int j = i + 1; j < ctarr; j++) {
            Clients[j - 1] = Clients[j];
         }
         ctarr--;
         ArrayResize(Clients, ctarr);           
      }
   }
}

// ---------------------------------------------------------------------
// Termination - clean up sockets
// ---------------------------------------------------------------------

void OnDeinit(const int reason)
{
   closesocket(ServerSocket);
   
   for (int i = 0; i < ArraySize(Clients); i++) {
      delete Clients[i];
   }
   ArrayResize(Clients, 0);
   
   #ifdef COMPILE_AS_EA
   EventKillTimer();
   CreatedTimer = false;
   #endif
}


// ---------------------------------------------------------------------
// Simple wrapper around each connected client socket
// ---------------------------------------------------------------------

class Connection {
private:
   // Client socket handle
   int mSocket;

   // Temporary buffer used to handle incoming data
   uchar mTempBuffer[SOCKET_READ_BUFFER_SIZE];
   
   // Stored-up data, waiting for a \r character 
   string mPendingData;
   
public:
   Connection(int ClientSocket);
   ~Connection();
   string GetID() {return IntegerToString(mSocket);}
   
   bool ReadAnyPendingData();
   void ProcessIncomingMessage(string strMessage);
};

// Constructor, called with the handle of a newly accepted socket
Connection::Connection(int ClientSocket)
{
   mPendingData = "";
   mSocket = ClientSocket; 
   
   // Put the client socket into non-blocking mode
   uint nbmode = 1;
   if (ioctlsocket(mSocket, 0x8004667E /* FIONBIO */, nbmode) != 0) {Print("ERROR in setting non-blocking mode on client socket");}
}

// Destructor. Simply close the client socket
Connection::~Connection()
{
   closesocket(mSocket);
}

// Called repeatedly on a timer, to check whether any
// data is available on this client connection. Returns true if the 
// client still seems to be connected (*not* if there's new data); 
// returns false if the connection seems to be dead. 
bool Connection::ReadAnyPendingData()
{
   // Check the client socket for data-readability
   timeval waitfor;
   waitfor.secs = 0;
   waitfor.usecs = 0;

   fd_set PollClientSocket;
   PollClientSocket.count = 1;
   PollClientSocket.single_socket = mSocket;

   int selres = select(0, PollClientSocket, 0, 0, waitfor);
   if (selres > 0) {
      
      // Winsock says that there is data waiting to be read on this socket
      int res = recv(mSocket, mTempBuffer, SOCKET_READ_BUFFER_SIZE, 0);
      if (res > 0) {
         // Convert the buffer to a string, and add it to any pending
         // data which we already have on this connection
         string strIncoming = CharArrayToString(mTempBuffer, 0, res);
         mPendingData += strIncoming;
         
         // Do we have a complete message (or more than one) ending in \r?
         int idxTerm = StringFind(mPendingData, "\r");
         while (idxTerm >= 0) {
            if (idxTerm > 0) {
               string strMsg = StringSubstr(mPendingData, 0, idxTerm);         
               
               // Strip out any \n characters lingering from \r\n combos
               StringReplace(strMsg, "\n", "");
               
               // Print the \r-terminated message in the log
               ProcessIncomingMessage(strMsg);
            }               
         
            // Keep looping until we have extracted all the \r delimited 
            // messages, and leave any residue in the pending data 
            mPendingData = StringSubstr(mPendingData, idxTerm + 1);
            idxTerm = StringFind(mPendingData, "\r");
         }
         
         return true;
      
      } else if (WSAGetLastError() == 10035 /* WSAEWOULDBLOCK */) {
         // Would block; not an error
         return true;
         
      } else {
         // recv() failed. Assume socket is dead
         return false;
      }
   
   } else if (selres == -1) {
      // Assume socket is dead
      return false;
      
   } else {
      // No pending data
      return true;
   }
}

// Can override this with whatever you want to do with incoming messages:
// handling trading commands, send data back down the socket etc etc etc
void Connection::ProcessIncomingMessage(string strMessage)
{
   Print("#" , GetID() , ": " , strMessage);
}

MQL5 Wizard: Development of Flexible MetaTrader 5 trading MetaTrader 5 Automations

MariopNew 2016.09.21 18:24 #23

jjc:

For a bit of fun...

Lol. Amazing. Congrats. I didn't though it could be possible to code these functionalities on MQL.

JC 2016.09.21 18:40 #24

Mariop:

Lol. Amazing. Congrats. I didn't though it could be possible to code these functionalities on MQL.

My instinct is always to do server sockets using multi-threading, but I can't actually think of a reason why this code wouldn't be viable and safe for real-life usage.

There's one thing in the most recent version, above, which isn't ideal. Rather than posting yet another large block of code, I'll describe it instead:

The use of WSASyncSelect() to do the faster event-based handling depends on having a chart window handle.
But attempting to use CHART_WINDOW_HANDLE will fail in OnInit() if MT4 is starting up with an EA already and previously attached to a chart (like the fact, already handled, that EventSetMillisecondTimer can fail in OnInit)
Therefore, if you restart MT4 with the EA already attached and running, the call to WSAAsyncSelect will fail, and you will only get timer-based handling of the socket activity, not the faster event-driven handling.

It's a small change to the code, but I don't want to post yet another big block into this forum when only you are interested.

MQL5 Wizard: Development of When MetaTrader 5 Web MetaTrader 5 Automations

JC 2016.09.22 09:54 #25

jjc:

Rather than posting yet another large block of code [...]

On second thoughts...

Yet another version, with the following changes:

Works on MT5 (both 32-bit and 64-bit) as well as MT4
Handles the fact that CHART_WINDOW_HANDLE may fail in OnInit
Does faster processing of the event-driven notifications via OnChartEvent, only checking the specific socket which generated the event rather than running the entire MainLoop()
Optionally allows connections from any computer rather than just localhost, switchable via a parameter in Inputs

// *********************************************************************
// Example of non-blocking server sockets, allowing connections
// from multiple concurrent clients. This example waits for 
// each \r-terminated line from a client (e.g. Telnet), and prints it 
// to the Experts log. The behaviour can be extended into something
// more realistic by changing Connection::ProcessIncomingMessage().
//
// Works on both MT4 and MT5 (32-bit and 64-bit). But see the 
// notes to socketselect3264() about the cheat which is used
// on 64-bit MT5.
// *********************************************************************

#property strict 


// ---------------------------------------------------------------------
// This code can either create an EA or a script. To create a script,
// comment out the line below. The code will then have OnTimer() 
// and OnTick() instead of OnStart()
// ---------------------------------------------------------------------

#define COMPILE_AS_EA


// ---------------------------------------------------------------------
// If we are compiling as a script, then we want the script
// to display the inputs page
// ---------------------------------------------------------------------

#ifndef COMPILE_AS_EA
#property show_inputs
#endif


// ---------------------------------------------------------------------
// User-configurable parameters
// ---------------------------------------------------------------------

// Purely cosmetic; causes MT4 to display the comments below
// as the options for the input, instead of using a boolean true/false
enum AllowedAddressesEnum {
   aaeLocal = 0,   // localhost only (127.0.0.1)
   aaeAny = 1      // All IP addresses
};

// User inputs
input int                     PortNumber = 51234;     // TCP/IP port number
input AllowedAddressesEnum    AcceptFrom = aaeLocal;  // Accept connections from


// ---------------------------------------------------------------------
// Constants
// ---------------------------------------------------------------------

// Size of temporary buffer used to read from client sockets
#define SOCKET_READ_BUFFER_SIZE  10000

// Defines the frequency of the main-loop processing, in milliseconds.
// This is either the duration of the sleep in OnStart(), if compiled
// as a script, or the frequency which is given to EventSetMillisecondTimer()
// if compiled as an EA. (In the context of an EA, the use of additional
// event-driven handling means that the timer is only a fallback, and
// this value could be set to a larger value, just acting as a sweep-up)
#define SLEEP_MILLISECONDS       10


// ---------------------------------------------------------------------
// Forward definitions of classes
// ---------------------------------------------------------------------

// Wrapper around a connected client socket
class Connection;

// ---------------------------------------------------------------------
// Winsock structure definitions and DLL imports
// ---------------------------------------------------------------------

struct sockaddr_in {
   short af_family;
   short port;
   int addr;
   int dummy1;
   int dummy2;
};

struct timeval {
   int secs;
   int usecs;
};

struct fd_set {
   int count;
   int single_socket;
   int dummy[63];
};

struct fd_set64 {
   long count;
   long single_socket;
   long dummy[63];
};

#import "Ws2_32.dll"
   int socket(int, int, int);   
   int bind(int, sockaddr_in&, int);
   int htons(int);
   int listen(int, int);
   int accept(int, int, int);
   int closesocket(int);
   int select(int, fd_set&, int, int, timeval&);
   int select(int, fd_set64&, int, int, timeval&);  // See notes to socketselect3264() below
   int recv(int, uchar&[], int, int);
   int ioctlsocket(int, uint, uint&);
   int WSAGetLastError();
   int WSAAsyncSelect(int, int, uint, int);
#import   


// ---------------------------------------------------------------------
// Global variables
// ---------------------------------------------------------------------

// Flags whether OnInit was successful
bool SuccessfulInit = false;

// Handle of main listening server socket
int ServerSocket;

// List of currently connected clients
Connection * Clients[];

#ifdef COMPILE_AS_EA
// For EA compilation only, we track whether we have 
// successfully created a timer
bool CreatedTimer = false;   

// For EA compilation only, we track whether we have 
// done WSAAsyncSelect(), which can't reliably done in OnInit()
// because a chart handle isn't necessarily available
bool DoneAsyncSelect = false;
#endif

// ---------------------------------------------------------------------
// Initialisation - create listening socket
// ---------------------------------------------------------------------

void OnInit()
{
   SuccessfulInit = false;
   
   if (!TerminalInfoInteger(TERMINAL_DLLS_ALLOWED)) {Print("Requires \'Allow DLL imports\'");return;}
   
   // (Don't need to call WSAStartup because MT4 must have done this)

   // Create the main server socket
   ServerSocket = socket(2 /* AF_INET */, 1 /* SOCK_STREAM */, 6 /* IPPROTO_TCP */);
   if (ServerSocket == -1) {Print("ERROR " , WSAGetLastError() , " in socket creation");return;}
   
   // Put the socket into non-blocking mode
   uint nbmode = 1;
   if (ioctlsocket(ServerSocket, 0x8004667E /* FIONBIO */, nbmode) != 0) {Print("ERROR in setting non-blocking mode on server socket");return;}
   
   // Bind the socket to the specified port number. In this example,
   // we only accept connections from localhost
   sockaddr_in service;
   service.af_family = 2 /* AF_INET */;
   service.addr = (AcceptFrom == aaeAny ? 0 /* INADDR_ANY */ : 0x100007F /* 127.0.0.1 */);
   service.port = (short)htons(PortNumber);
   if (bind(ServerSocket, service, 16 /* sizeof(service) */) == -1) {Print("ERROR " , WSAGetLastError() , " in socket bind");return;}

   // Put the socket into listening mode
   if (listen(ServerSocket, 10) == -1) {Print("ERROR " , WSAGetLastError() , " in socket listen");return;}

   #ifdef COMPILE_AS_EA
   // If we're operating as an EA, set up event-driven handling of the sockets
   // as described below. (Also called from OnTick() because this requires a 
   // window handle, and the set-up will fail here if MT4 is starting up
   // with the EA already attached to a chart.)
   SetupAsyncSelect();
   #endif

   // Flag that we've successfully initialised
   SuccessfulInit = true;

   // If we're operating as an EA rather than a script, then try setting up a timer
   #ifdef COMPILE_AS_EA
   CreateTimer();
   #endif   
}


// ---------------------------------------------------------------------
// Variation between EA and script, depending on COMPILE_AS_EA above.
// The script version simply calls MainLoop() from a loop in OnStart().
// The EA version sets up a timer, and then calls MainLoop from OnTimer().
// It also has event-driven handling of socket activity by getting
// Windows to simulate keystrokes whenever a socket event happens.
// This is even faster than relying on the timer.
// ---------------------------------------------------------------------

#ifdef COMPILE_AS_EA
   // .........................................................
   // EA version. Simply calls MainLoop() from a timer.
   // The timer has previously been created in OnInit(), 
   // though we also check this in OnTick()...
   void OnTimer()
   {
      MainLoop();
   }
   
   // The function which creates the timer if it hasn't yet been set up
   void CreateTimer()
   {
      if (!CreatedTimer) CreatedTimer = EventSetMillisecondTimer(SLEEP_MILLISECONDS);
   }

   // Timer creation can sometimes fail in OnInit(), at least
   // up until MT4 build 970, if MT4 is starting up with the EA already
   // attached to a chart. Therefore, as a fallback, we also 
   // handle OnTick(), and thus eventually set up the timer in
   // the EA's first tick if we failed to set it up in OnInit().
   // Similarly, the call to WSAAsyncSelect() may not be possible
   // in OnInit(), because it requires a chart handle, and that does
   // not exist in OnInit() during MT4 start-up.
   void OnTick()
   {
      CreateTimer();
      SetupAsyncSelect();
   }

   // In the context of an EA, we can improve the latency even further by making the 
   // processing event-driven, rather than just on a timer. If we use
   // WSAAsyncSelect() to tell Windows to send a WM_KEYDOWN whenever there's socket
   // activity, then this will fire the EA's OnChartEvent below. We can thus collect
   // socket events even faster than via the timed check, which becomes a back-up.
   // Note that WSAAsyncSelect() on a server socket also automatically applies 
   // to all client sockets created from it by accept(), and also that
   // WSAAsyncSelect() puts the socket into non-blocking mode, duplicating what
   // we have already above using ioctlsocket().
   // The further complication is that WSAAsyncSelect() requires a window handle,
   // and this is not available in OnInit() during MT4 start-up. Therefore,
   // like the timer though for different reasons, we repeat the call to
   // this function during OnTick()
   void SetupAsyncSelect()
   {
      if (DoneAsyncSelect) return;
      if (WSAAsyncSelect(ServerSocket, (int)ChartGetInteger(0, CHART_WINDOW_HANDLE), 0x100 /* WM_KEYDOWN */, 0xFF /* All events */) == 0) {
         DoneAsyncSelect = true;
      }
   }

   // In an EA, the use of WSAAsyncSelect() above means that Windows will fire a key-down 
   // message whenever there is socket activity. We can then respond to KEYDOWN events
   // in MQL4 as a way of knowing that there is socket activity to be dealt with.
   void OnChartEvent(const int id, const long& lparam, const double& dparam, const string& sparam)
   {
      if (id == CHARTEVENT_KEYDOWN) {
         // For a pseudo-keypress from WSAAsyncSelect(), the lparam will be the socket handle
         // (and the dparam will be the type of socket event, e.g. dparam = 1 = FD_READ).
         // Of course, this event can also be fired for real keypresses on the chart...
         
         // What is the lparam?
         if (lparam == ServerSocket) {
            // New pending connection on the listening server socket
            AcceptNewConnections();
         
         } else {
            // See if lparam is one of the client sockets, by looping 
            // through the Clients[] array
            int ctarr = ArraySize(Clients);
            for (int i = ctarr - 1; i >= 0; i--) {
               if (Clients[i].SocketHandle() == lparam) {

                  // Yes, we have found a socket matching this "keyboard" event.
                  // Return value from ReadAnyPendingData() is true
                  // if the socket still seems to be alive; false if 
                  // the connection seems to have been closed, and should be discarded
                  if (Clients[i].ReadAnyPendingData()) {
                     // Socket still seems to be alive
                     
                  } else {
                     // Socket appears to be dead
                     ReleaseClientSocket(i, ctarr);
                  }
                  return; //Early exit!
               }
            }
            
            // If we get here, then the lparam does not match any
            // of the sockets, and the event must be a real keypress,
            // not a pseudo-keypress from WSAAsyncSelect()
         }
      }
   }
    
#else

   // .........................................................
   // Script version. Simply calls MainLoop() repeatedly
   // until the script is removed, with a small sleep between calls
   void OnStart()
   {
      while (!IsStopped()) {
         MainLoop();
         Sleep(SLEEP_MILLISECONDS);
      }
   }
#endif


// ---------------------------------------------------------------------
// Main processing loop which handles new incoming connections, and reads
// data from existing connections.
// Can either be called from OnTimer() in an EA; or from OnStart() in a script, 
// on a continuous loop until IsStopped() is true. In an EA, socket 
// activity should almost always get handled by the event-driven stuff 
// above, and this timer loop is just a fallback.
// ---------------------------------------------------------------------

void MainLoop()
{
   // Do nothing if init was unsuccessful
   if (!SuccessfulInit) return;

   // This main timer loop does two things: accepts new incoming 
   // connections, and reads pending data on all sockets (including
   // new ones which have just been accepted)
   AcceptNewConnections();
   ReadSocketMessages();   
}

// Accept any new pending connections on the main listening server socket,
// wrapping the new caller in a Connection object and putting it
// into the Clients[] array
void AcceptNewConnections()
{
   int selres = socketselect3264(ServerSocket);
   if (selres > 0) {
   
      Print("New incoming connection...");
      int NewClientSocket = accept(ServerSocket, 0, 0);
      if (NewClientSocket == -1) {
         if (WSAGetLastError() == 10035 /* WSAEWOULDBLOCK */) {
            // Blocking warning; ignore
            Print("... would block; ignore");
         } else {
            Print("ERROR " , WSAGetLastError() , " in socket accept");
         }

      } else {
         Print("...accepted");
         
         // Create a new Connection object to wrap the client socket,
         // and add it to the Clients[] array
         int ctarr = ArraySize(Clients);
         ArrayResize(Clients, ctarr + 1);
         Clients[ctarr] = new Connection(NewClientSocket);               
         Print("Got connection to client #", Clients[ctarr].GetID());
      }
   }
}


// Process any incoming data from all client connections
// (including any which have just been accepted, above)
void ReadSocketMessages()
{
   int ctarr = ArraySize(Clients);
   for (int i = ctarr - 1; i >= 0; i--) {
      // Return value from ReadAnyPendingData() is true
      // if the socket still seems to be alive; false if 
      // the connection seems to have been closed, and should be discarded
      if (Clients[i].ReadAnyPendingData()) {
         // Socket still seems to be alive
         
      } else {
         // Socket appears to be dead. Delete, and remove from list
         ReleaseClientSocket(i, ctarr);
      }
   }
}

// Discards a client socket which appears to have died, deleting the Connection class
// and removing it from the Clients[] array. Takes two parameters: the index of 
// the connection within the Clients[] array, and the current size of that array (which
// is passed by reference, with the function sending back the decremented size)
void ReleaseClientSocket(int idxReleaseAt, int & SizeOfArray)
{
   Print("Lost connection to client #", Clients[idxReleaseAt].GetID());
   
   delete Clients[idxReleaseAt];
   for (int j = idxReleaseAt + 1; j < SizeOfArray; j++) {
      Clients[j - 1] = Clients[j];
   }
   SizeOfArray--;
   ArrayResize(Clients, SizeOfArray);           
}

// Wraps the Winsock select() function, doing an immediate non-blocking
// check of a single socket. This is the one area where we must
// vary things between 32-bit and 64-bit. On 64-bit MT5, we can get away
// with using Winsock calls where everything is defined as a 32-bit int;
// we can rely on the fact that socket and window handles are only ever going
// to be 4 bytes rather than 8 bytes - naughty, but works in practice.
// The one exception is the fd_set structure. On 64-bit MT5, 
// we must pass a version of the fd_set where everything is defined
// as 8-byte long rather than 4-byte int. Note that this applies to
// the MT5 version, not the O/S. On 32-bit MT5, even on a 64-bit computer,
// we use the same structure as MT4: 4-byte int rather than 8-byte long
int socketselect3264(int TestSocket)
{
   timeval waitfor;
   waitfor.secs = 0;
   waitfor.usecs = 0;

   // The following, incidentally, is permitted on MT4, but obviously returns false...
   if (TerminalInfoInteger(TERMINAL_X64)) {
      fd_set64 PollSocket64;
      PollSocket64.count = 1;
      PollSocket64.single_socket = TestSocket;

      return select(0, PollSocket64, 0, 0, waitfor);

   } else {
      fd_set PollSocket32;
      PollSocket32.count = 1;
      PollSocket32.single_socket = TestSocket;
   
      return select(0, PollSocket32, 0, 0, waitfor);
   }
}


// ---------------------------------------------------------------------
// Termination - clean up sockets
// ---------------------------------------------------------------------

void OnDeinit(const int reason)
{
   closesocket(ServerSocket);
   
   for (int i = 0; i < ArraySize(Clients); i++) {
      delete Clients[i];
   }
   ArrayResize(Clients, 0);
   
   #ifdef COMPILE_AS_EA
   EventKillTimer();
   CreatedTimer = false;
   DoneAsyncSelect = false;
   #endif
}


// ---------------------------------------------------------------------
// Simple wrapper around each connected client socket
// ---------------------------------------------------------------------

class Connection {
private:
   // Client socket handle
   int mSocket;

   // Temporary buffer used to handle incoming data
   uchar mTempBuffer[SOCKET_READ_BUFFER_SIZE];
   
   // Stored-up data, waiting for a \r character 
   string mPendingData;
   
public:
   Connection(int ClientSocket);
   ~Connection();
   string GetID() {return IntegerToString(mSocket);}
   int SocketHandle() {return mSocket;}
      
   bool ReadAnyPendingData();
   void ProcessIncomingMessage(string strMessage);
};

// Constructor, called with the handle of a newly accepted socket
Connection::Connection(int ClientSocket)
{
   mPendingData = "";
   mSocket = ClientSocket; 
   
   // Put the client socket into non-blocking mode
   uint nbmode = 1;
   if (ioctlsocket(mSocket, 0x8004667E /* FIONBIO */, nbmode) != 0) {Print("ERROR in setting non-blocking mode on client socket");}
}

// Destructor. Simply close the client socket
Connection::~Connection()
{
   closesocket(mSocket);
}

// Called repeatedly on a timer, to check whether any
// data is available on this client connection. Returns true if the 
// client still seems to be connected (*not* if there's new data); 
// returns false if the connection seems to be dead. 
bool Connection::ReadAnyPendingData()
{
   // Check the client socket for data-readability
   int selres = socketselect3264(mSocket);
   if (selres > 0) {
      
      // Winsock says that there is data waiting to be read on this socket
      int szData = recv(mSocket, mTempBuffer, SOCKET_READ_BUFFER_SIZE, 0);
      if (szData > 0) {
         // Convert the buffer to a string, and add it to any pending
         // data which we already have on this connection
         string strIncoming = CharArrayToString(mTempBuffer, 0, szData);
         mPendingData += strIncoming;
         
         // Do we have a complete message (or more than one) ending in \r?
         int idxTerm = StringFind(mPendingData, "\r");
         while (idxTerm >= 0) {
            if (idxTerm > 0) {
               string strMsg = StringSubstr(mPendingData, 0, idxTerm);         
               
               // Strip out any \n characters lingering from \r\n combos
               StringReplace(strMsg, "\n", "");
               
               // Print the \r-terminated message in the log
               ProcessIncomingMessage(strMsg);
            }               
         
            // Keep looping until we have extracted all the \r delimited 
            // messages, and leave any residue in the pending data 
            mPendingData = StringSubstr(mPendingData, idxTerm + 1);
            idxTerm = StringFind(mPendingData, "\r");
         }
         
         return true;
      
      } else if (WSAGetLastError() == 10035 /* WSAEWOULDBLOCK */) {
         // Would block; not an error
         return true;
         
      } else {
         // recv() failed. Assume socket is dead
         return false;
      }
   
   } else if (selres == -1) {
      // Assume socket is dead
      return false;
      
   } else {
      // No pending data
      return true;
   }
}

// Can override this with whatever you want to do with incoming messages:
// handling trading commands, send data back down the socket etc etc etc
void Connection::ProcessIncomingMessage(string strMessage)
{
   Print("#" , GetID() , ": " , strMessage);
}

MetaTrader 5 Built-in Trading MetaTrader Market - the MetaTrader 5 for your

Fernando Carreiro 2016.09.22 10:05 #26

jjc: On second thoughts...

Instead of posting such long SRC contributions, consider just attaching the file itself instead. I would suggest you EDIT your longer posts, deleting the SRC sections and just attaching the source file.

JC 2016.09.22 10:10 #27

FMIC:

Thank you very much for your contribution.

But, among the many reasons why I'm not going to do this is the fact that the code above works on both MT4 and MT5. If I uploaded it as an attachment, it would misleadingly have to be flagged as either .mq4 or .mq5 whereas in fact it works as both.

How to become a MQL5 Wizard: Development of Subscribe to a trade

JC 2016.09.22 10:48 #28

jjc:

But, among the many reasons [...]

Another of the key reasons is the following scenario, which could have applied to the OP:

" I know quite a bit about socket programming in a language such as Java. I wonder if anyone has written a socket server for MT4? If they have, then it must involve using functions such as listen(), bind(), htons() etc. So I will try to find such an article by doing a Google search for 'mql4 bind listen htons' "

As far as I am aware, neither search engines nor the search on this site can look inside attachments. So, posting the code as an attachment would make it invisible to a search such as "mql4 bind listen htons".

You regularly tell people that they should have done a search or consulted the documentation; I am trying to help with exactly that.

(As it happens, "mql4 bind listen htons" doesn't find this topic because Google hasn't crawled the page since September 14th. But it should start appearing on Google soon.)

MetaEditor - Professional editor Download MetaTrader 5 build MetaTrader Market - the

Fernando Carreiro 2016.09.22 10:51 #29

jjc:

Thank you very much for your contribution.

It that case make it into an include header file (".mqh"). That way it will be reusable in both MQL4 and MQL5. Just make sure to not embed the code directly in the event handlers, OnInit(), OnDeinit(), etc. - Give those functions separate and unique names, and then the user just has to place a call to them in there own code from those handlers.

MetaEditor - Professional editor MQL5 Wizard: Development of How to Start a

MariopNew 2016.09.22 21:14 #30

jjc:

On second thoughts...

Yet another version, with the following changes:

Works on MT5 (both 32-bit and 64-bit) as well as MT4
Handles the fact that CHART_WINDOW_HANDLE may fail in OnInit
Does faster processing of the event-driven notifications via OnChartEvent, only checking the specific socket which generated the event rather than running the entire MainLoop()
Optionally allows connections from any computer rather than just localhost, switchable via a parameter in Inputs

THANKS again jjc,

Right now I'm in a hurry and don't have time to test it, but I'll do it next week and tell you about. I think I won't have any problem running it, but maybe I'll ask you some doubts. BTW: Please don't edit any post; they all are perfect as they are (I feel like in StackOverflow =)).

When MetaTrader 5 Web MetaTrader Market - the Download MetaTrader 5 build

1 2 3 4 5

New comment