Deals History inside OnTesterPass

[Ricardo Luceac]

Hi,


I'm trying to develop  a Walk Forward test in my EA, but i'm running in a problem.

I can get the statistics of the pass inside the  OnTesterPass() via frameadd, but i cant pass an array of object to frameadd..

Then I try to get HistoryDeals, but it returns a lot of deals, but for the symbol I'm using on test returns nothing.

How can I manage to get the history of deals that are performed in a strategy test pass?


Thanks...

[fxsaber]

https://www.mql5.com/en/code/18801

[Alain Verleyen]

Ricardo Luceac:
Hi,


I'm trying to develop  a Walk Forward test in my EA, but i'm running in a problem.

I can get the statistics of the pass inside the  OnTesterPass() via frameadd, but i cant pass an array of object to frameadd..

Then I try to get HistoryDeals, but it returns a lot of deals, but for the symbol I'm using on test returns nothing.

How can I manage to get the history of deals that are performed in a strategy test pass?


Thanks...

What do you mean, what deals are returned if not from your test ?

Please show your code if you need coding help.

[fxsaber]

Alain Verleyen:

What do you mean, what deals are returned if not from your test ?

From real-account.

[fxsaber]

Форум по трейдингу, автоматическим торговым системам и тестированию торговых стратегий

Вопросы от начинающих MQL5 MT5 MetaTrader 5

fxsaber, 2017.08.23 14:10

// Пример записи данных Агентов (включая Облачные) в один файл
input int Range = 0;

void OnTick()
{
// ....
}

// Файл открываем только в режимах одиночночного прогона или Фрейма.
const int handle = ((MQLInfoInteger(MQL_TESTER) && !MQLInfoInteger(MQL_OPTIMIZATION)) || MQLInfoInteger(MQL_FRAME_MODE)) ?
                   FileOpen(__FILE__, FILE_WRITE | FILE_TXT) : INVALID_HANDLE;

// Подготовка данных
void GetData( string &Str, MqlTick &Ticks[], double &Balance )
{
  Str = "Hello World!";
  
  CopyTicks(_Symbol, Ticks, COPY_TICKS_ALL, 0, 2); // Последние два тика (пример)
  
  Balance = AccountInfoDouble(ACCOUNT_BALANCE);
}

// Запись данных
void SaveData( const string &Str, const MqlTick &Ticks[], const double Balance )
{
  FileWrite(handle, Str);
  
  for (int i = 0; i < ArraySize(Ticks); i++)
    FileWrite(handle, Ticks[i].bid);
    
  FileWrite(handle, Balance);
}

void OnTesterDeinit()
{
  if (handle != INVALID_HANDLE)  
    FileClose(handle);
    
  ChartClose();
}

#include <TypeToBytes.mqh> // https://www.mql5.com/ru/code/16280

double OnTester()
{
  string Str;
  MqlTick Ticks[];
  double Balance;
  
  GetData(Str, Ticks, Balance); // Подготовка данных для записи

  if (MQLInfoInteger(MQL_OPTIMIZATION)) // Оптимизация
  {
    CONTAINER<uchar> Container; // https://www.mql5.com/ru/forum/95447/page4#comment_5464205
    
    Container[0] = Str;
    Container[1] = Ticks;
    Container[2] = Balance;
  
    FrameAdd(NULL, 0, 0, Container.Data); // Отправили данные из Агента на Терминал
  }
  else // Одиночный прогон
  {    
    if (handle != INVALID_HANDLE)
      SaveData(Str, Ticks, Balance); // Данные будут записаны в MQL5\Files-папку Агента (не Терминала)
    
    FileClose(handle);
  }
  
  return(0);
}

void OnTesterPass()
{    
  if (handle != INVALID_HANDLE)
  {
    ulong Pass;
    string Name;
    long ID;
    double Value;
  
    CONTAINER<uchar> Container; // https://www.mql5.com/ru/forum/95447/page4#comment_5464205
  
    while (FrameNext(Pass, Name, ID, Value, Container.Data))
    {
      string Str;
      MqlTick Ticks[];
      double Balance;
      
      // Получили данные от Агента
      Container[0].Get(Str);
      Container[1].Get(Ticks);
      Container[2].Get(Balance);
      
//      FileWrite(handle, Pass);     // Если хочется записать номер прохода
      SaveData(Str, Ticks, Balance); // Данные будут записаны в MQL5\Files-папку Терминала (не Агента)
    }
  }
}

[Ricardo Luceac]

I manage to get statistical data with the frameadd, but it only acceps primitive types as the array input.

I try to create a class to pass to frameadd, so I can get History of Deals in framenext, but I get an error saying that the type is not correct.(even if i create the object as an array)

If i can pass an object to the frameadd, i think getting history there will be the right deals...

//--- Statistical parameters
double stat_values[8]; // Array for testing parameters
double best_result;
string best_parameter[];
CArrayObj* wf_data; 
CTesterInfo tester_info;

//+------------------------------------------------------------------+
//|                                                                  |
//+------------------------------------------------------------------+
double OnTester()
{
    if(GravaResultados||WF_Enable)
     {
       GetTestStatistics(stat_values);
     
        FrameAdd("Statistics",1,0,stat_values);

         //GetFrameStatistics(tester_info);
 //      FrameAdd("Deals",2,0,tester_info); //Gives error "'FrameAdd' - no one of the overloads can be applied to the function call"

      }

Then, the way I try to get the deals are as follows, but it returns a lot of deals, much more than the test ones... I think even the test deals are not returned...

//+------------------------------------------------------------------+
//|                                                                  |
//+------------------------------------------------------------------+
void OnTesterPass()
  { 
//--- If writing of optimization results is enabled
  // if(GravaResultados)
  //   {
       static int passes_count=0;                // Pass counter
   int        parameters_count=0;            // Number of Expert Advisor parameters
   int        optimized_parameters_count=0;  // Counter of optimized parameters
   string     string_to_write="";            // String for writing
   bool       include_criteria_list=false;   // For determining the start of the list of parameters/criteria
   int        equality_sign_index=0;         // The '=' sign index in the string
   string     name            ="";           // Public name/frame label
   ulong      pass            =0;            // Number of the optimization pass at which the frame is added
   long       id              =0;            // Public id of the frame
   double     value           =0.0;          // Single numerical value of the frame
   string     parameters_list[];             // List of the Expert Advisor parameters of the "parameterN=valueN" form
//--- Increase the pass counter
   passes_count++;
//--- Place statistical values into the array
   FrameNext(pass,name,id,value,stat_values);
//--- Get the pass number, list of parameters, number of parameters
   FrameInputs(pass,parameters_list,parameters_count);
//--- Iterate over the list of parameters in a loop (starting from the upper one on the list)
//    The list starts with the parameters that are flagged for optimization
     
      // Print("Pass: ","",pass);
      double res = TesterFunction(TestCost,TestType,Volume,TestCostContrato,TestCostOper,stat_values[0],stat_values[1],
                                  stat_values[2],stat_values[3],stat_values[4]);
      if(best_result==0)
      {
         best_result=res;
         ArrayCopy(best_parameter,parameters_list);
         Print("Best Result: "+DoubleToString(best_result,2));
      }
      else
      {
         if(best_result<res)
         {
             best_result=res;
             ArrayCopy(best_parameter,parameters_list);
             Print("Best Result: "+DoubleToString(best_result,2));
         }
      }            
  // }
   
   if(WF_Enable)
   {
      Print("WF_ENABLE");
     // MqlDateTime wfo_dt_start;
    //  MqlDateTime wfo_dt_end;
      
      if(WF_StartDate >= WF_EndDate)
         return;
      
 //     TimeToStruct(WF_StartDate,wfo_dt_start);      
  //    TimeToStruct(WF_EndDate,wfo_dt_end);      

      
      
      string wf_splited[];
      StringSplit(WF_testes,':',wf_splited);
      Print("WF Splited: " + wf_splited[0] + " - " + wf_splited[1]);
      
      int wf_in_window=StringToInteger(wf_splited[0]);
      int wf_out_window=StringToInteger(wf_splited[1]);
      int window_index = 0;
      int wf_full_window = wf_in_window+wf_out_window;
      
//      if((wfo_dt_start.year==wfo_dt_end.year)&&(wfo_dt_start.day_of_year+(wf_in_window+wf_out_window)>wfo_dt_end.day_of_year))
 //        return;
   
    if((WF_StartDate+(wf_full_window*24*60*60))>WF_EndDate)
      return;
   
   
    int qtd_windows = CalcWFWindows(WF_StartDate,WF_EndDate,wf_in_window,wf_out_window);
     
      
      
     Print("WF Qtd Windows: "+qtd_windows); 
   
      for(int i=0;i<qtd_windows;i++)
      {
         datetime in_window_start;
         datetime in_window_end;
         datetime out_window_start;
         datetime out_window_end;
         double in_profit=0;
         double out_profit=0;

      
        if(i==0)
        {
             in_window_start = WF_StartDate;
             in_window_end = (in_window_start+(wf_in_window*24*60*60))-1;
             out_window_start = in_window_end+1;
             out_window_end = (out_window_start+(wf_out_window*24*60*60))-1;
        
         }
         else
         {
             in_window_start = WF_StartDate+(i*(wf_out_window*24*60*60));
             in_window_end = (in_window_start+(wf_in_window*24*60*60))-1;
             out_window_start = in_window_end+1;
             out_window_end = (out_window_start+(wf_out_window*24*60*60))-1;
            
         }
         
         
         HistorySelect(in_window_start,in_window_end);
         uint     in_total=HistoryDealsTotal();
         Print("Total number of IN orders:"+in_total);
         for(uint i=0;i<in_total;i++)
        {
            ulong    ticket=0;
          //--- tentar obter ticket negócios
          if((ticket=HistoryDealGetTicket(i))>0)
          {
               //--- obter as propriedades negócios            
               double   price;
               double   profit;
               datetime time;
               string   symbol;
               long     type;
               long     entry;
               datetime dt;

 
 
               price =HistoryDealGetDouble(ticket,DEAL_PRICE);
               time  =(datetime)HistoryDealGetInteger(ticket,DEAL_TIME);
               symbol=HistoryDealGetString(ticket,DEAL_SYMBOL);
              type  =HistoryDealGetInteger(ticket,DEAL_TYPE);
               entry =HistoryDealGetInteger(ticket,DEAL_ENTRY);
               profit=HistoryDealGetDouble(ticket,DEAL_PROFIT);
               dt=HistoryDealGetInteger(ticket,DEAL_TIME);
 
               //--- apenas para o símbolo atual
               
            //   Print("Deal:"+ticket+" Symbol:"+symbol+" Profit:"+DoubleToString(profit,2)+" Data:"+dt); 
               if(symbol== Symbol_Tester)
                  in_profit+=profit;
        }
     }
       
       
          HistorySelect(out_window_start,out_window_end);
         uint     out_total=HistoryDealsTotal();
         Print("Total number of OUT deals:"+out_total);
         for(uint i=0;i<out_total;i++)
        {
            ulong    ticket=0;
          //--- tentar obter ticket negócios
          if((ticket=HistoryDealGetTicket(i))>0)
          {
               //--- obter as propriedades negócios            
               double   price;
               double   profit;
               datetime time;
               string   symbol;
               long     type;
               long     entry;
               datetime dt;
               price =HistoryDealGetDouble(ticket,DEAL_PRICE);
               time  =(datetime)HistoryDealGetInteger(ticket,DEAL_TIME);
               symbol=HistoryDealGetString(ticket,DEAL_SYMBOL);
               type  =HistoryDealGetInteger(ticket,DEAL_TYPE);
               entry =HistoryDealGetInteger(ticket,DEAL_ENTRY);
               profit=HistoryDealGetDouble(ticket,DEAL_PROFIT);
               dt=HistoryDealGetInteger(ticket,DEAL_TIME);
               //--- apenas para o símbolo atual
               
            //   Print("Deal:"+ticket+" Symbol:"+symbol+" Profit:"+DoubleToString(profit,2)+" Data:"+dt); 
                if(symbol== Symbol_Tester)
                  out_profit+=profit;
        }
     }
       
         Print("Window "+i+" IN from "+in_window_start+" to "+in_window_end+" Profit:"+DoubleToString(in_profit,2));
         Print("             OUT from "+out_window_start+" to "+out_window_end+" Profit:"+DoubleToString(out_profit,2));  
      }
   
   }
   
 }
  

Thanks!

[Alain Verleyen]

Ricardo Luceac:

I manage to get statistical data with the frameadd, but it only acceps primitive types as the array input.

I try to create a class to pass to frameadd, so I can get History of Deals in framenext, but I get an error saying that the type is not correct.(even if i create the object as an array)

If i can pass an object to the frameadd, i think getting history there will be the right deals...

Then, the way I try to get the deals are as follows, but it returns a lot of deals, much more than the test ones... I think even the test deals are not returned...

Thanks!

Ok I see, I don't have much time to check now. But what you could do with FrameAdd() is to use file instead of an array of objects ?

[Ricardo Luceac]

Yes, I think of that, but I don't wanna use anything external....

[Anthony Garot]

I can get the statistics of the pass inside the  OnTesterPass() via frameadd, but i cant pass an array of object to frameadd..

I avoided using frames. There were some limitations like the passing of only an array of doubles that had me switch tactics.

What worked for me:

I invoke optimizations externally through the command line (via python script). I dump all the statistics that I need to a CSV file from the OnTester() event.

That is to say, for every single optimization pass:

• Open the .csv file in append mode.
• Dump a single line
• Close the file

Also, I wanted the ability to manage dependent input parameters during the process. I ran into difficulties doing this in MQL5, so now I simply create a new .set file on-the-fly based upon previous results.

Then I try to get HistoryDeals, but it returns a lot of deals, but for the symbol I'm using on test returns nothing.

I've never tried getting deal information in OnTesterPass(). With my approach, I simply get deal information on-the-fly in OnTradeTransaction(). No need for looping . . . just handle the deal events as they come in.

[Ricardo Luceac]

I've never tried getting deal information in OnTesterPass(). With my approach, I simply get deal information on-the-fly in OnTradeTransaction(). No need for looping . . . just handle the deal events as they co

Ok, but how do you know in what Pass of the optimization generated the deal?

Do you create a .set file for each test pass, and dont use the optimization at all?

Thanks for the reply.

[Anthony Garot]

Ricardo Luceac:

Ok, but how do you know in what Pass of the optimization generated the deal?

Do you create a .set file for each test pass, and dont use the optimization at all?

Thanks for the reply.

See my comment here:

https://www.mql5.com/en/forum/35683#comment_6397172

I so use the optimizer, yes, and blend my results (CSV file) with the optimizer's results (XML file).

Python is nice for automating all of this because it has libraries for handling CSV, XML, and even INI files.