Trademinator 3: Rise of the Trading Machines

Roman Zamozhnyy | 27 February, 2012

Prologue

Once upon a time on a faraway forum (MQL5) two articles: "Genetic Algorithms - It's Easy!" by joo and "Dr. Tradelove..." by me were published. In the first article the author equipped us with a powerful tool for optimizing whatever you need, including trading strategies - a genetic algorithm implemented by means of the MQL5 language.

Using this algorithm, in the second article I tried to develop a self-optimizing Expert Advisor based on it. The article ended with the formulation of the following task: to create an Expert Advisor (self-optimizing, of course), which can not only select the best parameters for a particular trading system, but also choose the best strategy of all developed strategies. Let's see whether it is possible, and if it is, then how.

Tales of Trading Robots

First, we formulate the general requirements for a self-optimizing Expert Advisor.

It should be able to (based on historical data):

• select the best strategy of described ones
• choose the best financial instrument
• choose the best deposit size for trading with leverage correction
• choose the best parameters of indicators in the selected strategy

Further, in real life, it should be able to:

• open and close positions
• choose the size of the position
• decide on whether a new optimization is needed

The below figure shows a schematic diagram of the proposed Expert Advisor.

A detailed scheme with bounds is in the attached file Scheme_en.

Keeping in mind that it is impossible to grasp the immensity, we introduce restrictions in the Expert Advisor logic. We agree that (IMPORTANT):

1. The Expert Advisor will make trade decisions upon the occurrence of a new bar (on any timeframe that we select).
2. On the basis of p.1, but not limited to, the Expert Advisor will close trades only on the indicator signals not using Take Profit and Stop Loss and, accordingly, not using Trailing Stop.
3. The condition to start a new optimization: a drawdown of the balance is higher than the preset value during initialization of the level. Please note that this is my personal condition, and each of you can select your specific condition.
4. A fitness function models trading on the history and maximizes the modeled balance, provided that the relative drawdown of the balance of the simulated trades is below a certain preset level. Also note that this is my personal fitness function, and you can select your specific one.
5. We limit the number of parameters to be optimized, except for the three general ones (strategy, instrument and deposit share), to five for the parameters of indicator buffers. This limitation follows logically from the maximum number of indicator buffers for built-in technical indicators. If you are going to describe the strategies that use custom indicators with a large number of indicator buffers, simply change the OptParamCount variable in the main.mq5 file to the desired amount.

Now that the requirements are specified and limitations selected, you can look at the code that implements all this.

Let's start with the function, where everything runs.

void OnTick()
{
  if(isNewBars()==true)
  {
    trig=false;
    switch(strat)
    {
      case  0: {trig=NeedCloseMA()   ; break;};                      //The number of case strings must be equal to the number of strategies
      case  1: {trig=NeedCloseSAR()  ; break;};
      case  2: {trig=NeedCloseStoch(); break;};
      default: {trig=NeedCloseMA()   ; break;};
    }
    if(trig==true)
    {
      if(GetRelDD()>maxDD)                                           //If a balance drawdown is above the max allowed value:
      {
        GA();                                                        //Call the genetic optimization function
        GetTrainResults();                                           //Get the optimized parameters
        maxBalance=AccountInfoDouble(ACCOUNT_BALANCE);               //Now count the drawdown not from the balance maximum...
                                                                     //...but from the current balance
      }
    }
    switch(strat)
    {
      case  0: {trig=NeedOpenMA()   ; break;};                       //The number of case strings must be equal to the number of strategies
      case  1: {trig=NeedOpenSAR()  ; break;};
      case  2: {trig=NeedOpenStoch(); break;};
      default: {trig=NeedOpenMA()   ; break;};
    }
    Print(TimeToString(TimeCurrent()),";","Main:OnTick:isNewBars(true)",
          ";","strat=",strat);
  }
}

What is here? As drawn in the diagram, we look at each tick, whether there is a new bar. If there is, then, knowing what strategy is now chosen, we call its specific function for checking if there is an open position and close it, if necessary. Suppose now the best breakthrough strategy is SAR, respectively, the NeedCloseSAR function will be called:

bool NeedCloseSAR()
{
  CopyBuffer(SAR,0,0,count,SARBuffer);
  CopyOpen(s,tf,0,count,o);
  Print(TimeToString(TimeCurrent()),";","StrategySAR:NeedCloseSAR",
        ";","SAR[0]=",SARBuffer[0],";","SAR[1]=",SARBuffer[1],";","Open[0]=",o[0],";","Open[1]=",o[1]);
  if((SARBuffer[0]>o[0]&&SARBuffer[1]<o[1])||
     (SARBuffer[0]<o[0]&&SARBuffer[1]>o[1]))
  {
    if(PositionsTotal()>0)
    {
      ClosePosition();
      return(true);
    }
  }
  return(false);
}

Any position closing function must be boolean and return true when closing a position. This allows the next code block of the OnTick() function to decide on whether a new optimization is needed:

    if(trig==true)
    {
      if(GetRelDD()>maxDD)                                           //If the balance drawdown is above the max allowed one:
      {
        GA();                                                        //Call the genetic optimization function
        GetTrainResults();                                           //Get optimized parameters
        maxBalance=AccountInfoDouble(ACCOUNT_BALANCE);                   //Now count the drawdown not from the balance maximum...
                                                                     //...but from the current balance
      }
    }

Get the current balance drawdown and compare it with the maximum allowed one. If it has exceeded the max value, run a new optimization (GA()). The GA() function, in turn, calls the heart of the Expert Advisor - the fitness function FitnessFunction(int chromos) of the GAModule.mqh module:

void FitnessFunction(int chromos)                                    //A fitness function for the genetic optimizer:...
                                                                     //...selects a strategy, symbol, deposit share,...
                                                                     //...parameters of indicator buffers;...
                                                                     //...you can optimize whatever you need, but...
                                                                     //...watch carefully the number of genes
{
  double ff=0.0;                                                     //The fitness function
  strat=(int)MathRound(Colony[GeneCount-2][chromos]*StratCount);     //GA selects a strategy
 //For EA testing mode use the following code...
  z=(int)MathRound(Colony[GeneCount-1][chromos]*3);                  //GA selects a symbol
  switch(z)
  {
    case  0: {s="EURUSD"; break;};
    case  1: {s="GBPUSD"; break;};
    case  2: {s="USDCHF"; break;};
    case  3: {s="USDJPY"; break;};
    default: {s="EURUSD"; break;};
  }
//..for real mode, comment the previous code and uncomment the following one (symbols are selected in the MarketWatch window)
/*
  z=(int)MathRound(Colony[GeneCount-1][chromos]*(SymbolsTotal(true)-1));//GA selects a symbol
  s=SymbolName(z,true);
*/
  optF=Colony[GeneCount][chromos];                                   //GA selects a deposit share
  switch(strat)
  {
    case  0: {ff=FFMA(   Colony[1][chromos],                         //The number of case strings must be equal to the number of strategies
                         Colony[2][chromos],
                         Colony[3][chromos],
                         Colony[4][chromos],
                         Colony[5][chromos]); break;};
    case  1: {ff=FFSAR(  Colony[1][chromos],
                         Colony[2][chromos],
                         Colony[3][chromos],
                         Colony[4][chromos],
                         Colony[5][chromos]); break;};
    case  2: {ff=FFStoch(Colony[1][chromos],
                         Colony[2][chromos],
                         Colony[3][chromos],
                         Colony[4][chromos],
                         Colony[5][chromos]); break;};
    default: {ff=FFMA(   Colony[1][chromos],
                         Colony[2][chromos],
                         Colony[3][chromos],
                         Colony[4][chromos],
                         Colony[5][chromos]); break;};
  }
  AmountStartsFF++;
  Colony[0][chromos]=ff;
  Print(TimeToString(TimeCurrent()),";","GAModule:FitnessFunction",
        ";","strat=",strat,";","s=",s,";","optF=",optF,
        ";",Colony[1][chromos],";",Colony[2][chromos],";",Colony[3][chromos],";",Colony[4][chromos],";",Colony[5][chromos]);
}

Depending on the currently selected strategy, the fitness function calculation module, that is specific to a particular strategy, is called. For example, the GA has chosen a stochastic, FFStoch () will be called, and optimizing parameters of indicator buffers will be transfered to it:

double FFStoch(double par1,double par2,double par3,double par4,double par5)
{
  int    b;
  bool   FFtrig=false;                                               //Is there an open position?
  string dir="";                                                     //Direction of the open position
  double OpenPrice;                                                  //Position Open price
  double t=cap;                                                      //Current balance
  double maxt=t;                                                     //Maximum balance
  double aDD=0.0;                                                    //Absolute drawdown
  double rDD=0.000001;                                               //Relative drawdown
  Stoch=iStochastic(s,tf,(int)MathRound(par1*MaxStochPeriod)+1,
                         (int)MathRound(par2*MaxStochPeriod)+1,
                         (int)MathRound(par3*MaxStochPeriod)+1,MODE_SMA,STO_CLOSECLOSE);
  StochTopLimit   =par4*100.0;
  StochBottomLimit=par5*100.0;
  dig=MathPow(10.0,(double)SymbolInfoInteger(s,SYMBOL_DIGITS));
  leverage=AccountInfoInteger(ACCOUNT_LEVERAGE);
  contractSize=SymbolInfoDouble(s,SYMBOL_TRADE_CONTRACT_SIZE);
  b=MathMin(Bars(s,tf)-1-count-MaxMAPeriod,depth);
  for(from=b;from>=1;from--)                                         //Where to start copying of history
  {
    CopyBuffer(Stoch,0,from,count,StochBufferMain);
    CopyBuffer(Stoch,1,from,count,StochBufferSignal);
    if((StochBufferMain[0]>StochBufferSignal[0]&&StochBufferMain[1]<StochBufferSignal[1])||
       (StochBufferMain[0]<StochBufferSignal[0]&&StochBufferMain[1]>StochBufferSignal[1]))
    {
      if(FFtrig==true)
      {
        if(dir=="BUY")
        {
          CopyOpen(s,tf,from,count,o);
          if(t>0) t=t+t*optF*leverage*(o[1]-OpenPrice)*dig/contractSize; else t=0;
          if(t>maxt) {maxt=t; aDD=0;} else if((maxt-t)>aDD) aDD=maxt-t;
          if((maxt>0)&&(aDD/maxt>rDD)) rDD=aDD/maxt;
        }
        if(dir=="SELL")
        {
          CopyOpen(s,tf,from,count,o);
          if(t>0) t=t+t*optF*leverage*(OpenPrice-o[1])*dig/contractSize; else t=0;
          if(t>maxt) {maxt=t; aDD=0;} else if((maxt-t)>aDD) aDD=maxt-t;
          if((maxt>0)&&(aDD/maxt>rDD)) rDD=aDD/maxt;
        }
        FFtrig=false;
      }
   }
    if(StochBufferMain[0]>StochBufferSignal[0]&&StochBufferMain[1]<StochBufferSignal[1]&&StochBufferMain[1]>StochTopLimit)
    {
      CopyOpen(s,tf,from,count,o);
      OpenPrice=o[1];
      dir="SELL";
      FFtrig=true;
    }
    if(StochBufferMain[0]<StochBufferSignal[0]&&StochBufferMain[1]>StochBufferSignal[1]&&StochBufferMain[1]<StochBottomLimit)
    {
      CopyOpen(s,tf,from,count,o);
      OpenPrice=o[1];
      dir="BUY";
      FFtrig=true;
    }
  }
  Print(TimeToString(TimeCurrent()),";","StrategyStoch:FFStoch",
        ";","K=",(int)MathRound(par1*MaxStochPeriod)+1,";","D=",(int)MathRound(par2*MaxStochPeriod)+1,
        ";","Slow=",(int)MathRound(par3*MaxStochPeriod)+1,";","TopLimit=",StochTopLimit,";","BottomLimit=",StochBottomLimit,
        ";","rDD=",rDD,";","Cap=",t);
  if(rDD<=trainDD) return(t); else return(0.0);
}

The fitness function of the stochastic returns a simulated balance to the main function, which will pass it to the genetic algorithm. At some point in time the GA decides to end the optimization, and using the GetTrainResults() function, we return the best current values of the strategy (for example - moving averages), symbol, the deposit share and parameters of the indicator buffers to the basic program, as well as create indicators for further real trading:

void GetTrainResults()                                               //Get the best parameters
{
  strat=(int)MathRound(Chromosome[GeneCount-2]*StratCount);          //Remember the best strategy
//For EA testing mode use the following code...
  z=(int)MathRound(Chromosome[GeneCount-1]*3);                       //Remember the best symbol
  switch(z)
  {
    case  0: {s="EURUSD"; break;};
    case  1: {s="GBPUSD"; break;};
    case  2: {s="USDCHF"; break;};
    case  3: {s="USDJPY"; break;};
    default: {s="EURUSD"; break;};
  }
//...for real mode, comment the previous code and uncomment the following one (symbols are selected in the MarketWatch window)
/*
  z=(int)MathRound(Chromosome[GeneCount-1]*(SymbolsTotal(true)-1));  //Remember the best symbol
  s=SymbolName(z,true);
*/
  optF=Chromosome[GeneCount];                                        //Remember the best deposit share
  switch(strat)
  {
    case  0: {GTRMA(   Chromosome[1],                                //The number of case strings must be equal to the number of strategies
                       Chromosome[2],
                       Chromosome[3],
                       Chromosome[4],
                       Chromosome[5]) ; break;};
    case  1: {GTRSAR(  Chromosome[1],
                       Chromosome[2],
                       Chromosome[3],
                       Chromosome[4],
                       Chromosome[5]) ; break;};
    case  2: {GTRStoch(Chromosome[1],
                       Chromosome[2],
                       Chromosome[3],
                       Chromosome[4],
                       Chromosome[5]) ; break;};
    default: {GTRMA(   Chromosome[1],
                       Chromosome[2],
                       Chromosome[3],
                       Chromosome[4],
                       Chromosome[5]) ; break;};
  }
  Print(TimeToString(TimeCurrent()),";","GAModule:GetTrainResults",
        ";","strat=",strat,";","s=",s,";","optF=",optF,
        ";",Chromosome[1],";",Chromosome[2],";",Chromosome[3],";",Chromosome[4],";",Chromosome[5]);
}

void GTRMA(double par1,double par2,double par3,double par4,double par5)
{
  MAshort=iMA(s,tf,(int)MathRound(par1*MaxMAPeriod)+1,0,MODE_SMA,PRICE_OPEN);
  MAlong =iMA(s,tf,(int)MathRound(par2*MaxMAPeriod)+1,0,MODE_SMA,PRICE_OPEN);
  CopyBuffer(MAshort,0,from,count,ShortBuffer);
  CopyBuffer(MAlong, 0,from,count,LongBuffer );
  Print(TimeToString(TimeCurrent()),";","StrategyMA:GTRMA",
        ";","MAL=",(int)MathRound(par2*MaxMAPeriod)+1,";","MAS=",(int)MathRound(par1*MaxMAPeriod)+1);
}

Now it all is back to the place where everything is running (OnTick()): knowing what strategy is now the best one, it is checked whether it is time to go to the market:

bool NeedOpenMA()
{
  CopyBuffer(MAshort,0,0,count,ShortBuffer);
  CopyBuffer(MAlong, 0,0,count,LongBuffer );
  Print(TimeToString(TimeCurrent()),";","StrategyMA:NeedOpenMA",
        ";","LB[0]=",LongBuffer[0],";","LB[1]=",LongBuffer[1],";","SB[0]=",ShortBuffer[0],";","SB[1]=",ShortBuffer[1]);
  if(LongBuffer[0]>LongBuffer[1]&&ShortBuffer[0]>LongBuffer[0]&&ShortBuffer[1]<LongBuffer[1])
  {
    request.type=ORDER_TYPE_SELL;
    OpenPosition();
    return(false);
  }
  if(LongBuffer[0]<LongBuffer[1]&&ShortBuffer[0]<LongBuffer[0]&&ShortBuffer[1]>LongBuffer[1])
  {
    request.type=ORDER_TYPE_BUY;
    OpenPosition();
    return(false);
  }
  return(true);
}

The circle closed up.

Let's check how it works. Here is a 2011 report on the 1-hour timeframe with four major pairs: EURUSD, GBPUSD, USDCHF, USDJPY: