Interaction between MеtaTrader 4 and MATLAB Engine (Virtual MATLAB Machine)

Introduction

MetaTrader 4 and the MATLAB mathematical package have gained high popularity among users due to their positive characteristics, including "flexibility" in creation of complex calculation systems. There are three main ways of MATLAB connection with external applications, but only one of them is recommended - use of the virtual desktop MATLAB Engine. This method guarantees the full compatibility with the whole MATLAB package. Many programmers avoid this method for below reasons:

• Many users find it slow. This is true, if compared with the direct function call from DLL libraries of MATLAB. The main delay takes place at the beginning of operation, when the virtual machine is called because of the call of numerous libraries that are uploaded into the virtual space of the calling process (in our case MetaTrader 4).
• Transferability of the project. True, when transferring a project to another computer all MATLAB DLL libraries must also be transferred, though as well as when direct call is used, to know the "relations" of the latter ones, i.e. start queue!
• Obligatory knowledge of C++ or Fortran. Well, if you know MQL4, you can easily learn C++ and vice versa.

Why I recommend this method:

1. This is the most reliable and independent from the MATLAB version method of connection with external programs. You can change MATLAB version and your indicators or Expert Advisors won't notice it. This is the most important advantage.
2. It has a relatively quick development method. It doesn't require debuggers, and it will make no difficulties to write the DLL wrapper.
3. "Common desktop" for several indicators and/or Expert Advisors. I consider this method useful when we need to make a decision based on data of several indicators or in the implementation of a pyramid trade.

This article describes the way of connecting MetaTrader 4 and MATLAB ver. 7.4.0 (R2007a) via a "DLL-wrapper" written in Borland C++ Builder 6. Programmers who prefer Microsoft products will have to adapt examples to their compiler (good luck to you in this complicated matter!).

I. Setting a Task

First of all we need to define what should we start the project from. Let's divide the development process into three parts:

1. Development of the M-function in MATLAB that implements the calculation of an indicator/EA.
2. Development of the "DLL-wrapper" to connect MATLAB and MetaTrader 4.
3. Development of the MQL program.

II. Developing the M-Function

This is probably the most interesting and long-running process that includes the following actions:

1. Pre-export of data from MetaTrader 4 into MATLAB.

Figures show the process of the manual data export into MATLAB. When the export is over, variables will be created in the MATLAB desktop.

2. Search for correct formulas, range of formula parameters, etc.

This process is creative and very important, but the development of the mathematical algorithm of an indicator and/or Expert Advisor is not the subject of our article. You can find information about it in literary about MATLAB.

3. M-function creation in MATLAB.

A programmer who knows C++ and/or MQL4 won't have any difficulties creating the function - moreover all variables have the sane data type - "matrix". I.e. it is not significant to clearly define a variable as an array or a multidimensional array - the language will do this itself. And I don't find the process of data type selection significant. As for me, I always use mxREAL. Well, perhaps more memory is used, but there isn't any confusion in such a case. More details can be found in references 1, 2. In the given example the filter of high frequencies is implemented.

III. Developing the "DLL-Wrapper"

Let's dwell on this point in mire details, because it is as ESSENTIAL as the air is. So, every DLL library of late binding must meet the following conditions:

• It must have internal functions for "waste" collection and memory clearing after its operation.
• It must be possibly multithreaded, i.e. support operation of more than one thread at the same time.
• It must be located in certain directories, see further: location of project files.

Main external functions of the "DLL-wrapper" are the API interface of MATLAB Engine and one function of the standard C++ input/output library. The API interface of MATLAB Engine is simple and compact; it contains only 8 functions:

Engine *pEng = engOpen(NULL) – function calling the MATLAB desktop, the parameter is always NULL, it returns pointer to the desktop descriptor, it is required for the operation of other functions, the variable is made global.

int exitCode = engClose(Engine *pEng) – function closing the desktop, pEng pointer to the desktop descriptor, returns value which is though unimportant, because this function is called at DLL closing and is not important, returns the number of MATLAB desktop "users".

mxArray *mxVector = mxCreateDoubleMatrix(int m, int n, int ComplexFlag) – the function creates a matrix for the MATLAB desktop, returns pointer to variable matrix. It is necessary for the creation of a variable compatible with MATLAB. Usual data arrays and/or simple data types cannot be sent to MATLAB!

mxArray *mxVector – pointer to variable matrix;

int m – number of rows;

int n – number of columns;

ComplexFlag – complex number type, always mxREAL for the correct operation with MetaTrader 4.

void = mxDestroyArray(mxArray *mxVector) – the function erases the MATLAB matrix, needed for clearing the memory. Always delete data when they are no more needed, otherwise there will be problems with memory or "overlapping" of results.

mxArray *mxVector – pointer to variable matrix.

int = engPutVariable( Engine *pEng, char *Name, mxArray *mxVector) – function sending a variable to the desktop. Variables of mxArray type must not only be created, but also sent to MATLAB.

Engine *pEng – pointer to desktop "descriptor";

char *Name – variable name in the MATLAB desktop, type - char;

mxArray *mxVector – pointer to variable matrix.

mxArray *mxVector = engGetVariable(Engine *pEng, char *Name) – function of variable receiving from the desktop, function opposite to the previous one. Variables of mxArray type can be received.

mxArray *mxVector – pointer to variable matrix;

Engine *pEng – pointer to desktop "descriptor";

char *Name – variable name in MATLAB desktop, type - char.

double *p = mxGetPr(mxArray *mxVector) – the function receives pointer to data array; it is used for copying data together with memcpy(…). Use this function when receiving/writing a variable of mxArray type, to extract/paste a variable of a simple type (int, double...).

double *p – pointer to the array of double type;

mxArray *mxVector – pointer to variable matrix.

int = engEvalString(Engine *pEng, char *Command) – function sends command to the desktop. The command in the Command line will be executed by the MATLAB desktop.

Engine *pEng – pointer to desktop "descriptor";

char *Command – command for MATLAB, line of char type.

There is only one function for working with memory:

void *pIn = memcpy (void *pIn, void *pOut, int nSizeByte) – function copying (cloning) a variable (array) pOut into pIn variable of nSizeByte byte size.

ATTENTION: Watch the array dimensionality. They must either be equal, or pIn array should be larger than pOut.

Requirements for Export Functions of "DLL-Wrapper"

For MetaTrader 4 to be able to use MATLAB, functions-transmitters should be written. Let's view requirements for the projecting of such functions. Any function that will be called from MetaTrader 4 must be __stdcall – i.e. parameters are transmitted via stack, the function clears the stack. This is how the function is declared:

extern "C" __declspec(dllexport) <variable_type> __stdcall Funcion(<type> <name>);

extern "C" __declspec(dllexport) - informs the C++ compiler that the function is external, it is written in export table.

<variable_type> - type of a variable to return; can be: void, bool, int, double, compound types and pointers cannot be transmitted; see further;

__stdcall – agreement on parameters transmission into the function and back;

Function – name of your function;

<type> <name> - type and name of the input variable; the maximal number of variables - 64.

Here is the prototype of function defining, also pay attention to __stdcall

bool __stdcall Funcion (<type> <name>)

{

//……

}

Beside this, a file with def extension should be created. Usually this is a text file that describes library name and names of export functions. If this file does not exist, your file will "think up" its own distorted function names which will complicate the use of DLL. Here is the file example:

LIBRARY NameDll

EXPORTS

NameFunctionA

NameFunctionB

LIBRARY – accessory word, points to DLL name.

EXPORTS – accessory word says that below function names will be enumerated.

NameFunctionA, NameFunctionB – names of DLL functions.

But there are restrictions imposed by MQL: Since this language does not have pointers, it has no dynamic memory, so arrays, structures, etc. cannot be passed from a DLL library. But in MetaTrader data can be written in arrays passed by a function by reference. The result can be written in an array created by MetaTrader, the pointer of which your DLL received. But the array must be of a certain dimensionality and cannot be an indicator line (this restriction is probably connected with the specific memory arrangement in MetaTrader 4).

Now, knowing how to write and what functions to call, let's view a typical algorithm of the "DLL-wrapper":

1. Starting MATLAB Engine using the engOpen() function during the first call of DLL;

2. Getting data from MetaTrader and sending back, DLL function;

2.1. Creating variables by the mxCreateDoubleMatrix() function;

2.2. Copying data into mxVector variable, functions memcpy() and mxGetPr();

2.3. Passing variables to MATLAB desktop, engPutVariable() function;

2.4. Passing formula/code to MATLAB desktop, engEvalString() function;

2.5. Receiving response from MATLAB desktop, engGetVariable() function;

2.6. Returning value to MetaTrader, functions memcpy() and mxGetPr();

3. Closing MATLAB using the engClose() function, deleting all variables mxDestroyArray() when loading DLL from the address area of MetaTrader process.

Now let's create the skeleton of the "DLL-wrapper":

/*---------------------------------------------------------------------------
** Libraries + *.lib + *.def:
** libeng.lib** libmx.lib
** libmex.lib** project_name.def
*/
#include <windows.h>#include <memory.h>#include "engine.h"
//---------------------------------------------------------------------------
extern "C" __declspec(dllexport)<variable_type>__stdcall Funcion(<type><name>);
//---------------------------------------------------------------------------
int WINAPI DllEntryPoint(HINSTANCE hinst,unsigned long reason,void *lpReserved)
  {
   /*    
** reason for DLL call    
*/
   switch(reason)
     {
      case DLL_PROCESS_ATTACH:
         /*            
** DLL uploaded into the address space of the process            
*/
         break;
      case DLL_PROCESS_DETACH:
         /*            
**DLL loaded from the address space of the process            
*/
         break;
     }
   return TRUE;
  }
//---------------------------------------------------------------------------
bool __stdcall Funcion(<type><name>)
  {
   ……
  }
//---------------------------------------------------------------------------

Project Assembly

The figure below shows how libraries and *.def files are added into the project:

Here is the list of files necessary for the "DLL-wrapper" project:

1. libeng.lib – located in: \Program Files\MATLAB\R2007a\extern\lib\win32\borland\
2. libmx.lib – located in: \Program Files\MATLAB\R2007a\extern\lib\win32\borland\
3. libmex.lib – located in: \Program Files\MATLAB\R2007a\extern\lib\win32\borland\
4. имя_проекта.def – this file should be created in a notepad as it is described above.

The engine.h file should be copied from \Program Files\MATLAB\R2007a\extern\\include into the folder \Program Files\Borland\CBuilder6\Include - thus you don't have to indicate path to the compiler every time.

Attention: These instructions are given for assembling the project in Borland C++ Builder 6 only!

IV. Developing a MQL4 Program

We will view questions connected only with the declaration of "DLL wrapper" functions and passing of parameters. So, in order to declare a function the below language construction is needed:

#import "HighPass.dll"

void ViewAnsFilter();

bool TestDllFilter();

bool AdaptiveHighFilter(double& nInVector[], int nSizeVector, double nSizeWind, double dAmplit);

void MakeBuffFilter(int nSize);

void DestrBuffFilter();

#import

where:

#import "HighPass.dll" – key word and the name of a DLL library;

void MakeBuffFilter(int nSize); - function name, type of a value to be returned, name and type of a passed value.

NB ! "[]" is used when passing arrays, the ampersand character "&" is necessary, if dll writes down a response into this data array! There are no other ways of array passing from external programs in MQL 4! The array to be passed must be of a certain dimensionality and cannot be an indicator array!

V. Location of Files

After the project building all project files should be correctly located:

*.dll and *.m - library files and m-functions in the catalog \Program Files\MetaTrader\experts\libraries;

*.mql is located in its usual place. i.e. if this is an indicator - in 'indicators' folder, if an EA - in 'experts', if a script - in 'scripts' folder.

Attention: When starting an indicator or Expert Advisor the warning about busy server can appear:

In such a case wait 5-10 seconds until Console Matlab appears in the taskbar and click "Repeat".

P.S. I have a notebook with 512 RAM, Celeron M 2100; I didn't experience any delays in the filter operation, number of charts - 5 with the total buffer 500 х 8 х 5 = 20 000 byte. So, the choice is up to you! As for me, I have already made it. If delays occur, a distributed calculation system can be easily implemented in MATLAB; i.e. several desktops can be started on different PCs connected into a local network.

Reference List

1. Built-in MATLAB Help.
2. "Matlab 5.х Calculations, Visualization, Programing" N.N. Martynov.
3. "C++ Builder 6. Reference Manual" A.Y. Arkhangelski.
4. Built-in MQL4 Help.

Conclusion

In this article we discussed the basics of "DLL-wrapper" development for binding MetaTrader 4 with the MATLAB mathematical package. We haven't touched upon questions of providing the operation of several indicators and/or Expert Advisors - they will be discussed in the next article. The attached file contains MACD improved due to the use of high frequency filter.