Download MetaTrader 5

Automated Optimization of a Trading Robot in Real Trading

4 October 2007, 12:06
Igor Malcev
31
8 913

Introduction

It is supposed that an Expert Advisor having inputs adjusted to the history will trade to a profit for the first (rather short) time. Indirect confirmations of this suggestion appeared after I had watched the Automated Trading Championship 2006. When the Championship started there were much more profitable Expert Advisors than later, when some of them turned to be noncompetitive. This is why I suppose the most of those Expert Advisors that had not come to the finish were adjusted to the history.

The idea to check this supposition in practice was born on the Russian forum of this website, in the section of Ideal Automated Trading System. The main idea is to start optimization of an EA automatically once a day and then analyze the obtained optimization results and record them in the EA's variables.

To implement this idea, we decided to take the ready-made Expert Advisor, MACD Sample, from the MetaTrader 4 Client Terminal and insert our own function of automated optimization into it. A bit later, the code of that automated optimizer was ready and uploaded in the same forum, in the section of Automated Optimizer. After some more time, the first confirmations of the idea appeared in the branch of Automated Optimizer. Later on, the optimizer was transformed into an mqh-library for better usability.


Installation of the Automated Optimizer

Below is what must be done for performing this task:

  • copy MACD Sample_1.mq4 into the 'expert' folder of the MetaTrader 4 Client Terminal preliminarily installed and connected to the Internet; and
  • copy the folder together with the installed MetaTrader 4 Client Terminal to a new location.

For better usability, we will hereinafter name the original terminal "Terminal", whereas the copy will be named "Terminal-Tester". We will perform a checking test on EURUSD at timeframe H1 using the Expert Advisor initially provided by the terminal, but slightly changed - MACD Sample_1.mq4.


Terminal-Tester Setup

Please don't forget to compile MACD Sample_1.mq4 in the Terminal-Tester. Let's start the client terminal first, then strategy tester, and set it up as shown in the screenshot below.

Optimization will be performed for three days. This is quite enough to check the automated optimizer. We will select the optimization date "From" according to the following formula - the current date minus three days. During optimization, the necessary history for the selected symbol (in our case, it is EURUSD) must be downloaded.

Those who perform optimization for the first time can find the description of necessary procedures in the Help menu of the MetaTrader 4 Client Terminal: <Help - Help Topics F1 - Auto Trading - Expert Optimization - Setup>. Or they can read the article named Testing of Expert Advisors in the MetaTrader 4 Client Terminal: An Outward Glance.

Then let's check variables to be optimized as shown in the screenshot below.

Automated optimization is limited to four variables, but it will be enough to have only three for our purposes and in order to save time. After the variables have been selected, let us store the optimization settings in the set-file named as MACD Sample_1.set. This file must be saved in the 'tester' folder of the Terminal-Tester. Then launch the pre-optimization and memorize the start time. This is necessary to calculate the time period needed for automated optimization with the preset parameters. After the optimization is finished, we will calculate the necessary waiting time. Then we have to close this terminal, since otherwise we won't be able to start it programmatically.


Setup of the Expert Advisor Located in the Terminal

For this, let's open the test Expert Advisor MACD Sample_1.mq4 in MetaEditor and perform the following:

- set the start time for auto optimization, for example, at 00:01 every day:

datetime SetHour    = 0;  // Optimization starting hour; 
        datetime SetMinute  = 1;  // Optimization starting minute.

- set the amount of days for optimization (it must be the same as that for pre-optimization):

int TestDay = 3;

- set the optimization end waiting time, in minutes, that we have calculated before, for example, 4 minutes:

int TimeOut = 4;

- type in the Expert Advisor name:

string NameMTS = "MACD Sample_1";  // EA's name

- type in the set-file name with the settings:

// Set-file name with the settings
        string NameFileSet = "MACD Sample_1.set";

- type in the path to the folder containing the installed Terminal-Tester, for instance:

// Path to the tester        
        string PuthTester = "D:\Program Files\Forex Best Trade Station";

- set the priority for filtering:

// Sorting by Maximal profit
        int Gross_Profit = 1;                      
        // Sorting by Maximal profit factor        
        int Profit_Factor = 2;     
        // Sorting by Maximal expected payoff
        int Expected_Payoff = 3;

- write variable names for optimization:

string Per1 = "FastEMA";
        string Per2 = "SlowEMA";
        string Per3 = "SignalSMA";
        string Per4 = "";

- copy attached file auto_optimization.mqh to the 'include' folder';

- include the library file in the Expert Advisor:

//--- Including the auto optimizer's library
#include <auto_optimization.mqh>

- it is only remains to copy the code below to the beginning of the start() function of your Expert Advisor. MACD Sample_1.mq4 contains it already.

// Not to be launched at testing and optimizing   
  if(!IsTesting() && !IsOptimization())
    {                
      // Compare the current hour with that preset for launching
      if(TimeHour(TimeLocal()) == SetHour)
        {
          // Protection against restarting
          if(!StartTest)
            {
              // Compare the minute range to the minute
              // preset for launching
              if(TimeMinute(TimeLocal()) > SetMinute - 1)
                {     
                  // the range is necessary, in case that for some reasons 
                  // no new tick is available for a long time
                  if(TimeMinute(TimeLocal()) < SetMinute + 1)
                    {  
                      // Flag of tester launching
                      StartTest    = true;
                      TimeStart    = TimeLocal();
                      Tester(TestDay, NameMTS, NameFileSet, PuthTester, 
                             TimeOut, Gross_Profit, Profit_Factor, 
                             Expected_Payoff, Per1, Per2, Per3, Per4);
                    
                    }
                }
            }
        }
    }
    
                        
   FastEMA      = GlobalVariableGet(Per1);
   SlowEMA      = GlobalVariableGet(Per2);
   SignalSMA    = GlobalVariableGet(Per3);
   TrailingStop = GlobalVariableGet(Per4);  
// If the tester launching is flagged  
  if(StartTest)
    {                                        
      // If more time has elapsed the launching than it was set 
      // to be the test waiting time
      if(TimeLocal() - TimeStart > TimeOut*60)
        {            
          // Zeroize the flag
          StartTest = false;                              
        }
    }

That's all. After the automated optimizer has been recompiled, it can be launched, but it may be used for only the same symbol and timeframe, for which the pre-optimization was performed. In our case, it is EURUSD on H1. To check the auto optimizer, you can insert the code given below in the int init () function, then the auto optimizer will be launched at the Expert Advisor's start.

Tester(TestDay,NameMTS,NameFileSet,PuthTester,TimeOut, Gross_Profit,Profit_Factor, 
       Expected_Payoff, Per1,Per2,Per3,Per4);


How Automated Optimizer Works

The automated optimizer works on the basis of using Terminal-Tester to optimize parameters of the Expert Advisor attached to the chart in Terminal. For this, the program sends to the Terminal-Tester a file containing optimization parameters (optimise.ini) and launches the Terminal Tester in the optimization mode. Then it copies the obtained results of "FileReport........htm" back to the Terminal and filters the best values from the obtained results.


More Details about How Auto Optimizer Works

At the preset time, for example, at 00.01, the automated optimizer should be launched. Variables should be filled out with values.

// Path to the terminal 
string PuthTerminal = TerminalPath() + "\experts\files";
// Name of the ini file for the tester
string FileOptim    = "optimise.ini";
string FileOptim1   = "\optimise.ini";                                  
// Calculation of the starting date
datetime DayStart   = TimeLocal()-86400*TestDay;
// Optimization starting date
string DateStart    = TimeToStr(DayStart,TIME_DATE);
// Optimization ending date
string DateStop     = TimeToStr(TimeLocal(),TIME_DATE);
// Tester report file name
string FileReport   = "FileReport_" + Symbol() + "_" + DateStop + ".htm";
string FileReport1  = "\FileReport_" + Symbol() + "_" + DateStop + ".htm";
// Limitation for the minimum amount of trades per day
double MinTr        = TestDay - 2;
// Limitation for maximal amount of trades per day
double MaxTr        = (60 / Period()*TestDay) + 2;
// The amount of attempts to copy the report file
int    KvoPptk      = 10;
// The amount of lines for sorting
int    StepRes      = 12;

Then the parameters of the ini file are written in the string array:

// Prepare the ini file for optimization
ArrayOpttim[0] = ";optimise strategy tester";             
// Enable/Disable Expert Advisors
ArrayOpttim[1] = "ExpertsEnable = false";
// Name of the EA file
ArrayOpttim[2] = "TestExpert=" + NameMTS;
// Name of the file containing parameters
ArrayOpttim[3] = "TestExpertParameters=" + NameFileSet;
// Symbol
ArrayOpttim[4] = "TestSymbol=" + Symbol();
// Timeframe
ArrayOpttim[5] = "TestPeriod=" + Period();
// Modeling mode
ArrayOpttim[6] = "TestModel=" + 0;
// Recalculate
ArrayOpttim[7] = "TestRecalculate=false";
// Optimization
ArrayOpttim[8] = "TestOptimization=true";
// Use date
ArrayOpttim[9] = "TestDateEnable=true";
// From
ArrayOpttim[10] = "TestFromDate=" + DateStart;
// To
ArrayOpttim[11] = "TestToDate=" + DateStop;
// Report file name
ArrayOpttim[12] = "TestReport=" + FileReport;
// Rewrite the report file
ArrayOpttim[13] = "TestReplaceReport=true";
// Shut down the terminal upon completion
ArrayOpttim[14] = "TestShutdownTerminal=true";

The optimization parameters are recorded into the ini file from the array. You can also read about how to create an ini file in the MetaTrader 4 Client Terminal Help, see <Help - Help Topics F1 - Tools - Configuration at Startup>.

// Write data into the ini file                
// Find out about the array size
OptimArraySize = ArraySize(ArrayOpttim);
// Open a file to write
opttim = FileOpen(FileOptim, FILE_CSV|FILE_WRITE, 0x7F);
if(opttim > 0)
  {
    for(int i = 0; i < OptimArraySize; i++)
      {
        // from the array into the variable
        ini = ArrayOpttim[i];                                     
        // from the variable into the file
        FileWrite(opttim, ini);
      } 
    // close the file
    FileClose(opttim);
  }
else
  {
    Print("Failed writing data into the ini file. Error No ", 
          GetLastError());
    return(0);
  }

After the parameters have been recorded in the ini file, the shell32.dll included in the standard Windows delivery is connected and function ShellExecuteA is launched.

#import  "shell32.dll"               //Connect a dll (provided with Windows)       
  int ShellExecuteA(int hwnd,string Operation,string 
                    File,string Parameters,string Directory,int ShowCmd); 
#import

File containing the parameters will be sent to the Terminal Tester folder.

// copy the ini file into the tester folder 
copyini = ShellExecuteA(0,"Open","xcopy", "\"" + PuthTerminal + 
                        FileOptim1 + "\" \"" + PuthTester + "\" /y", 
                        "", 3);
// wait until the file is copied
Sleep(1200);                                                    
if(copyini < 0)
  {
    Print("Failed copying ini file");
    return(0);
  }

Then the tester is launched and starts to optimize the predefined variables. The Expert Advisor is in the halted state during optimization.

// Start Tester 
start = ShellExecuteA(0, "Open", "terminal.exe", FileOptim,
                      PuthTester, 3);
if(start < 0)
  {
    Print("Failed starting Tester");
    return(0);
  }
Comment("Wait until optimization is complete");
// wait until optimization is complete
Sleep(60000*TimeOut);

After optimization is complete, Tester will automatically record the results in the report file. That file is copied to the folder that contains the terminal.

for(Pptk = 0; Pptk < KvoPptk; Pptk++)
  {                    
    //Start a cycle attempting to compy the resport file
    Comment("Attempt # " + Pptk + " to copy the report file");
    ShellExecuteA(0, "Open", "xcopy", "\"" + PuthTester + FileReport1 + 
                  "\" \"" + PuthTerminal + "\" /y", "", 3);
    // wait until the file is copied
    Sleep(1200);
    // Try to open the report file
    file = FileOpen(FileReport, FILE_READ, 0x7F);
    if(file < 0)
      {
        // if it fails to open, wait some more and try again
        Sleep(60000);
      }                    
    else 
        break;             
  }
if(file < 0)
  {
    Print("Failed copying the report file");
    return(0);
  }

Then the data from the report file will be placed in the string array for further processing.

// Read from file into the array
// Cycle, until the file ends
while(FileIsEnding(file) == false)
  {                 
    // Read a string from the report file
    FileLine = FileReadString(file);
    // Find the necessary string and set the reference point there
    index = StringFind(FileLine, "title", 20);
    if(index > 0)
      {
        // Increase the array in size
        ArrayResize(ArrayStrg, NumStr + 1);
        // Record the strings from the file in the array
        ArrayStrg[NumStr] = FileLine;
        NumStr++;
      }
  }
// Close the file
FileClose(file);
// Delete the file in order not to produce too many copies
FileDelete(FileReport);
// Set the array size by the amount of data read from the file
ArrayResize(ArrayData, NumStr);

Then the necessary values are selected in the array:

for(text = 0; text < NumStr; text++)
     {
      select = ArrayStrg[text]; 
    //-------------------------------------------------------------------------
    //   Reporting text processing (These are apples and oranges)              | 
    //-------------------------------------------------------------------------
    // Position Pass 
    ClStep=StringFind(select, "; \">",20)+4;
    // Find the end of position
    ClStepRazm = StringFind(select, "td>", ClStep);
    // Read the value
    CycleStep = StringSubstr(select, ClStep, ClStepRazm - ClStep);
    // Position Profit 
    // Find the beginning of the position
    GrProf = StringFind(select, "", ClStepRazm);
    // Find the end of position
    GrProfRazm = StringFind(select, "td>", GrProf);
    // Read value
    GrossProfit = StringSubstr(select, GrProf+4, GrProfRazm - (GrProf + 4));
    // Position Total Trades
    // Find the beginning of position
    TotTrad = StringFind(select, "", GrProfRazm);
    // Find the end of position
    TotTradRazm = StringFind(select, "td>", TotTrad);
    // Read the value
    TotalTrades = StringSubstr(select, TotTrad+4, TotTradRazm - 
                               (TotTrad + 4));
    // Position Profitability
    // Find the beginning of position
    ProfFact = StringFind(select, "", TotTradRazm);
    // Find the end of position
    ProfFactRazm = StringFind(select, "td>", ProfFact);
    // Read the value
    ProfitFactor = StringSubstr(select, ProfFact + 4, ProfFactRazm - 
                                (ProfFact + 4));
    // Position Expected Payoff 
    // Find the beginning of position
    ExpPay = StringFind(select, "", ProfFactRazm);
    // Find the dn of position
    ExpPayRazm=StringFind(select, "td>", ExpPay);
    // Read the value
    ExpectedPayoff = StringSubstr(select, ExpPay+4, ExpPayRazm - 
                                  (ExpPay + 4));
    // Variables' positions starting with the second one
    // Find the beginning of position
    P1 = StringFind(select, Per1, 20);
    // Find the end of position
    P1k = StringFind(select, ";", P1);
    // Read the Variable
    Perem1 = StringSubstr(select, P1 + StringLen(Per1) + 1, P1k - 
                          (P1 + 1 + StringLen(Per1)));
    // Find the beginning of position
    P2 = StringFind(select, Per2, 20);
    // Find the end of position
    P2k = StringFind(select, ";", P2); 
    // Read the Variable
    Perem2 = StringSubstr(select, P2 + StringLen(Per2) + 1, P2k - 
                          (P2 + 1 + StringLen(Per2)));
    // Find the beginning of position
    P3 = StringFind(select, Per3, 20);
    // Find the end of position
    P3k = StringFind(select, ";", P3);
    // Read the Variable
    Perem3 = StringSubstr(select, P3 + StringLen(Per3) + 1, P3k - 
                          (P3 + 1 + StringLen(Per3)));
    // Find the beginning of position
    P4 = StringFind(select, Per4, 20);
    // Find the end of position
    P4k = StringFind(select, ";", P4);
    // Read the Variable 
    Perem4 = StringSubstr(select, P4 + StringLen(Per4) + 1, P4k - 
                          (P4 + 1 + StringLen(Per4)));
    Comment("The obtained results are being analyzed");

After that, the obtained results, before they are transformed into number format, have been filtered by the minimal and the maximal amount of trades. Zero in the value of Profit_Factor is replaced with 1000 for correct sorting and subsequent sifting.

// Transform into number format
TotalTradesTransit = StrToDouble(TotalTrades);
GrossProfitTransit = StrToDouble(GrossProfit);
ExpectedPayoffTran = StrToDouble(ExpectedPayoff);
nodubl = true;
if(MinTr < TotalTradesTransit && MaxTr > TotalTradesTransit)
  {                    
    // Filter by the amount of trades
    PrFactDouble = StrToDouble(ProfitFactor);
    // Replace 0 in the Profit_Factor for proper analysis
    if(PrFactDouble == 0)
      {
        PrFactDouble = 1000;
      }

Then the values are checked for duplicates and filtered out.

// Filter data having identical values
for(Dubl = 0; Dubl <= ResizeArayNew; Dubl++)
  {                    
    // Start the loop searching for identical values
    if(GrossProfitTransit == ArrayData[Dubl][1])
      {          
        // check whether the results for maximal profit coincide
        if(TotalTradesTransit == ArrayData[Dubl][2])
          {       
            // check whether the results for the amount of trades coincide
            if(PrFactDouble == ArrayData[Dubl][3])
              {          
                // check whether the results for Profit Factor coincide
                if(ExpectedPayoffTran == ArrayData[Dubl][4])
                  { 
                    // check whether the results for expected payoff coincide
                    nodubl=false;                              
                    // If everything coincides, flag it as coincided
                  }
              }
          }
      }
  }

Then the values prepared for sorting are written in the array.

// Write the filtered data in the array
if(nodubl)
  {
    ArrayData[text][1] = GrossProfitTransit;                                
    ArrayData[text][2] = TotalTradesTransit;
    ArrayData[text][3] = PrFactDouble;
    ArrayData[text][4] = ExpectedPayoffTran;
    ArrayData[text][5] = StrToDouble(Perem1);
    ArrayData[text][6] = StrToDouble(Perem2);
    ArrayData[text][7] = StrToDouble(Perem3);
    ArrayData[text][8] = StrToDouble(Perem4);
    ResizeArayNew++; 
  }

Then the data start to be analyzed in the preset priority order. The analysis is performed as follows:

  • the loop is launched and, at the first pass, the values are sorted by the first parameter, for instance, by maximal profit; several best values are selected (12, by default), others are cut off;
  • at the second pass, the values are sorted by the second parameter, for example, by Profit Factor; some best values are selected, a half after the first sorting, others are cut off;
  • at the third pass, the last sorting is performed for the third parameter, for example, by expected payoff; a half of values is taken after the second sorting, others are cut off.
// Analyzer
// Analyzing principle is the sequential checking of maximal 
// values according to the predefined filtering priority   
ArrayResize(ArrayTrans, ResizeArayNew - 1);
for(int PrioStep = 1; PrioStep < 4; PrioStep++)
  {
    for(PrCycle = 0; PrCycle < ResizeArayNew; PrCycle++)
      {
        Sort     = ArrayData[PrCycle][0];
        Prior1   = ArrayData[PrCycle][1];             
        transit  = ArrayData[PrCycle][2];
        Prior2   = ArrayData[PrCycle][3];             
        Prior3   = ArrayData[PrCycle][4];             
        transit1 = ArrayData[PrCycle][5];
        transit2 = ArrayData[PrCycle][6];
        transit3 = ArrayData[PrCycle][7];
        transit4 = ArrayData[PrCycle][8]; 
           
        if(PrioStep == 1)
          {
            //Prepare for the 1st sorting
            if(Gross_Profit ==1)
              {
                SortTrans = Prior1;
              }
            if(Profit_Factor == 1)
              {
                SortTrans = Prior2;
              }
            if(Expected_Payoff == 1)
              {
                SortTrans = Prior3;
              }
          }
        if(PrioStep == 2)
          {
            // Restore
            if(Gross_Profit ==1)
              {
                Prior1 = Sort;
              }
            if(Profit_Factor == 1)
              {
                Prior2 = Sort;
              }
            if(Expected_Payoff == 1)
              {
                Prior3 = Sort;
              } 
            //Prepare for the 2nd sorting
            if(Gross_Profit == 2)
              {
                SortTrans = Prior1;
              }
            if(Profit_Factor == 2)
              {
                SortTrans = Prior2;
              }
            if(Expected_Payoff == 2)
              {
                SortTrans = Prior3;
              }
          }
        if(PrioStep == 3)
          {
            // Restore
            if(Gross_Profit == 2)
              {
                Prior1 = Sort;
              }
            if(Profit_Factor == 2)
              {
                Prior2 = Sort;
              }
            if(Expected_Payoff == 2)
              {
                Prior3 = Sort;
              } 
            //Prepare for the 3rd sorting
            if(Gross_Profit ==3)
              {
                SortTrans = Prior1;
              }
            if(Profit_Factor == 3)
              {
                SortTrans = Prior2;
              }
            if(Expected_Payoff == 3)
              {
                SortTrans = Prior3;
              }
          }          
        ArrayTrans[PrCycle][0] = SortTrans;
        ArrayTrans[PrCycle][1] = Prior1;
        ArrayTrans[PrCycle][2] = transit;
        ArrayTrans[PrCycle][3] = Prior2;
        ArrayTrans[PrCycle][4] = Prior3;
        ArrayTrans[PrCycle][5] = transit1;
        ArrayTrans[PrCycle][6] = transit2;
        ArrayTrans[PrCycle][7] = transit3;
        ArrayTrans[PrCycle][8] = transit4;
      }
    ArraySort(ArrayTrans,StepRes, 0, MODE_DESCEND); // Sort the array
    ArrayResize(ArrayTrans, StepRes);               // Cut off the unnecessary things
    for(int CopyAr = 0; CopyAr < StepRes; CopyAr++)
      {
        ArrayData[CopyAr][0] = ArrayTrans[CopyAr][0];
        ArrayData[CopyAr][1] = ArrayTrans[CopyAr][1];
        ArrayData[CopyAr][2] = ArrayTrans[CopyAr][2];
        ArrayData[CopyAr][3] = ArrayTrans[CopyAr][3];
        ArrayData[CopyAr][4] = ArrayTrans[CopyAr][4];             
        // Per1    Variable 1
        ArrayData[CopyAr][5] = ArrayTrans[CopyAr][5];              
        // Per2    Variable 2
        ArrayData[CopyAr][6] = ArrayTrans[CopyAr][6];
        // Per3    Variable 3
        ArrayData[CopyAr][7] = ArrayTrans[CopyAr][7];
        // Per4    Variable 4
        ArrayData[CopyAr][8] = ArrayTrans[CopyAr][8];
      }
   StepRes = StepRes / 2;
  }

The values filtered in this manner are written in global variables. The values from global variables will be substituted in the EA.

// Write the obtained results in variables
   double Peremen1 = ArrayTrans[0][5];                         
   double Peremen2 = ArrayTrans[0][6];
   double Peremen3 = ArrayTrans[0][7];
   double Peremen4 = ArrayTrans[0][8];
   // If the variable name is specified, write the result in 
   // global variables
   if(Per1 != "")
     {
       GlobalVariableSet(Per1, Peremen1);
     }             
   if(Per2 != "")
     {
       GlobalVariableSet(Per2,Peremen2);
     }
   if(Per3 != "")
     {
       GlobalVariableSet(Per3,Peremen3);
     }
   if(Per4 != "")
     {
       GlobalVariableSet(Per4,Peremen4);
     }
   Comment(Per1, " ", Peremen1, "  | ", Per2, " ", Peremen2, "  | ", Per3, 
           " ", Peremen3, "  | ", Per4, " ", Peremen4);
   Print(Per1, " ", Peremen1, "  | ", Per2, " ", Peremen2, "  | ", Per3, 
         " ", Peremen3,"  | ",Per4," ",Peremen4);
  }  // Function ends. That's all, automated optimization is complete.


Auto Optimizer Operation Results

The operation results of the automated optimizer can be observed using messages that appear in the upper left corner of the chart as shown in the screenshot below:

Optimization Completeness Reference Time.

Analysis of Values Obtained after Optimization.

Resulting Values of Variables.

If the optimization results appear in the message, it means that optimization is complete and data have been received.

To estimate the automated optimizer's work independently, one may look through all files containing intermediate data and saved during the working process. Tester stores data in the file named "FileReport_EURUSD_2007.03. 12. htm" where the symbol and the date are substituted in the file name according to the selected symbol and the current date of optimization. This file can be found in the Terminal-Tester folder. These file with reports are not deleted automatically, so one may use them to check the changes of parameters.

The next file, FileTest1.csv, is saved after the values have been filtered by the amount of trades and the copies have been deleted. The file is saved in: D:\Program Files\terminal_folder_ame\experts\files

Then values obtained after each sifting step are saved to FileTest2.csv. The file is saved in this folder, too: D:\Program Files\terminal_folder_name\experts\files

The tables above show how the obtained values are filtered. The filtering order was set by default as: 1- Gross_Profit, 2- Profit_Factor, 3- Expected_Payoff.

The automated optimizer's code contains detailed comments and you can fit the most suitable variable parameters, if necessary. For example, you want to optimize your EA for a period of time other than the latest days, or you are planning to increase/decrease the amount of trades within the optimization period. For this, you just should change the corresponding variables directly in auto_optimization.mqh.

// Limitation of minimal amount of trades per day
double MinTr   = TestDay - 2; 
// Limitation on maximal amount of trades per day
double MaxTr   = (60 / Period()*TestDay) + 2;
// The amount of attempts to copy the report file
int    KvoPptk = 10;
// The amount of strings to be sorted
int    StepRes = 12;


Conclusion

This article is not aimed at teaching newbies elements of optimization, so it is highly recommended to learn normal optimization before setting up automated optimization of your Expert Advisor. It is better to use the automated optimizer after you have chosen the basic variables that will influence your Expert Advisor differently at different times. I.e., it is better to use this automated optimizer to fit parameters of the variables the changes in which influence the EA operation more than those in other variables, depending on the market volatility.

Besides, it is better not to set a very large automated optimization period. Suppose the Expert Advisor has been optimized for 6-12 hours every day. Then a question occurs: When will it trade? In other words, optimization is not necessary as itself. It is recommended to set optimization periodicity (the periodicity of optimizer launching is meant) considering the timeframe, on which the EA is supposed to trade. This means that it is necessary to consider that the historical data will be pumped when the Tester-Terminal is started and it is possible that the broker just does not possess the necessary historical data for the specified period of time. To verify the hypothesis described at the beginning of this article, you will need 24-hour and stable internet connection.

The automated optimization programs developed are located in the attached files: auto_optimization.mqh - the library itself, MACD Sample_1.mq4 - slightly changed Expert Advisor included in the MetaTrader 4 Client Terminal standard delivery set.

Translated from Russian by MetaQuotes Software Corp.
Original article: https://www.mql5.com/ru/articles/1467

Attached files |
MACD_Sample_1.mq4 (8.01 KB)
Last comments | Go to discussion (31)
ChuckM
ChuckM | 11 Feb 2011 at 03:25

Xeon,

I just found this and IT IS GREAT.

Have you rewritten auto_optimization_204.mqh to read the XML files that MT5 produces? If not, would you please consider doing it?

Thanks for your hard work.

ChuckM

ChuckM
ChuckM | 21 Feb 2011 at 14:44
ChuckM:

Xeon,

I just found this and IT IS GREAT.

Have you rewritten auto_optimization_204.mqh to read the XML files that MT5 produces? If not, would you please consider doing it?

Thanks for your hard work.

ChuckM


Nevermind. Using your code as a starting point, I rewrote auto-optimization to run on MT5. For anyone else who is considering taking on this task, here are a few of the larger issues I had to work through:

  • Tester no longer produces the output file when running optimization. But I was able to write my own file in the same format (so I could reuse more of Xeon's code). This file is created in the OnTester() event.
  • MQL5 doesn't support ArraySort() for multi-dimensional arrays. I had to write my own bubble sort. (Kind of a big deal since I'm not a c++ programmer.)
  • Something I haven't resolved yet...I need to add code to handle the condition that occurs when the optimizer can find no winning passes. Now it just unloads the EA.
  • Since I run my EA on multiple currencies, I added Symbol() as a prefix to all reports and Global Variables.

I'm really pleased with the result. Now to see if it improves the performance of my EA...

TIA...ChuckM

MQL4 Comments
MQL4 Comments | 14 Mar 2011 at 10:34

Three months ago, I would be sooo thankful for this article, it's great!! But I started with a full-auto-trader and I don't optimize anything at the moment :) Although, I restarted a bit with coding, just a bit to keep me fresh ;) and because I've the time now.

I downloaded your codes anyway. Thanks a lot for the ideas!

MQL4 Comments
MQL4 Comments | 1 Jan 2013 at 00:43
spgandau:
spgandau:

I am having trouble using this EA.  here is my info:

Windows 7 (64bit) home edition

MT4 v.4 build 445

I have set up both the the Terminal and Tester_terminal. I created the *.set files and placed them the tester folder.   noticed that there was a respective *.ini created as well by the pgm.  I set the Path to Testor as C:\Program Files (x86)\MT4_Tester.

I ran the optimizing routine on the terminal-tester for EURUSD using H1 for 3 most recent days.  After running this EA on Tester-Terminal, the optimise.ini was not created, nor were any reports generated.  

I was using #include <auto_optimization_204.mqh> and the downloaded MACDSample1.mq4 from above.

I feel lke I am missing something very obvious, but am quite stuck.  I would appreciate any input.

Regards,

spgandau

UPDATE 1 day later:  I finally figured out ho to get the optimise.ini properly created.  I did 2 things:

1. In Windows Explorer, I went to C:\Program Files (x86)\Terminal-Tester.  I RIGHT-Clicked on the folder and selected 'Properties'.  I then selected the 'General' tab.  At the bottom of the screen, the 'Attributes' check box for "Read-Only" was filled in solid.  I un-selected that check box, so that it was clear.  I then selected Apply.  Another pop-up box appeared to confirm my decision AND to "Apply changes to this folder, subfolders and files".  I clicked OK. 

2. THEN, inside the auto-optimize.mqh (include file), I changed the path to the Terminal-Tester.  The example above shows what appears to put the optimise.ini in the same folder as the terminal.exe (the root folder).  I changed the path of the Terminal-Tester to have the file created in  the \tester\files folder....and it worked!  Apparently, in MT4 you can create and place files in either the \experts\files or \tester\files areas or \history   (It will not work in for placement in any other folder).

The optimise.ini file was properly created in \tester\files folder.   

Now, my issue is to actually have data placed into that file (it was empty when I looked at it in Windows Explorer).  also, the reports are not being generated.




Quantum Capital International Group Ltd
Yang Chih Chou | 13 Jun 2014 at 15:12
Excuse me  ,some body can help me to use this method with MT4 Build 646, thank you. 
Step on New Rails: Custom Indicators in MQL5 Step on New Rails: Custom Indicators in MQL5

I will not list all of the new possibilities and features of the new terminal and language. They are numerous, and some novelties are worth the discussion in a separate article. Also there is no code here, written with object-oriented programming, it is a too serous topic to be simply mentioned in a context as additional advantages for developers. In this article we will consider the indicators, their structure, drawing, types and their programming details, as compared to MQL4. I hope that this article will be useful both for beginners and experienced developers, maybe some of them will find something new.

Here Comes the New MetaTrader 5 and MQL5 Here Comes the New MetaTrader 5 and MQL5

This is just a brief review of MetaTrader 5. I can't describe all the system's new features for such a short time period - the testing started on 2009.09.09. This is a symbolical date, and I am sure it will be a lucky number. A few days have passed since I got the beta version of the MetaTrader 5 terminal and MQL5. I haven't managed to try all its features, but I am already impressed.

Using text files for storing input parameters of Expert Advisors, indicators and scripts Using text files for storing input parameters of Expert Advisors, indicators and scripts

The article describes the application of text files for storing dynamic objects, arrays and other variables used as properties of Expert Advisors, indicators and scripts. The files serve as a convenient addition to the functionality of standard tools offered by MQL languages.

How to create an indicator of non-standard charts for MetaTrader Market How to create an indicator of non-standard charts for MetaTrader Market

Through offline charts, programming in MQL4, and reasonable willingness, you can get a variety of chart types: "Point & Figure", "Renko", "Kagi", "Range bars", equivolume charts, etc. In this article, we will show how this can be achieved without using DLL, and therefore such "two-for-one" indicators can be published and purchased from the Market.