Libraries: SingleTesterCache

[Automated-Trading]

SingleTesterCache:

Tester's single pass data.

Author: fxsaber

[fxsaber]

All statistics that are in the Single Run Report are available. Previously, much of this was impossible to get even if you have the source of the Expert Advisor.

  double            ghpr;                    // geometric mean transaction 
  double            ghprpercent;             // geometric mean transaction in per cent 
  double            ahpr;                    // arithmetic mean of the transaction 
  double            ahprpercent;             // arithmetic mean of transactions in per cent 
  double            zscore;                  // series test 
  double            zscorepercent;           // serial test in per cent 
  double            lrcorr;                  // correlation coefficient of linear regression 
  double            lrstderror;              // standard error of balance deviation from linear regression 
  double            corr_prf_mfe;            // correlation between mfe and profit 
  double            corr_prf_mae;            // correlation between mae and profit 
  double            corr_mfe_mae;            // correlation between mae and mfe 
  double            mfe_a;                   // correlation between mfe and profit, coefficient for linear regression line 
  double            mfe_b;                   // correlation between mfe and profit, coefficient for linear regression line 
  double            mae_a;                   // correlation between mae and profit, coefficient for linear regression line 
  double            mae_b;                   // correlation between mae and profit, coefficient for linear regression line 
  UINT              in_per_hours[24];        // distribution of inputs by hours 
  UINT              in_per_week_days[7];     // distribution of inputs by day of the week 
  UINT              in_per_months[12];       // distribution of inputs by month 
  double            out_per_hours[24][2];    // distribution of inputs by hours 
  double            out_per_week_days[7][2]; // distribution of inputs by day of the week 
  double            out_per_months[12][2];   // distribution of inputs by month 
  INT64             holding_time_min;        // minimum position holding time 
  INT64             holding_time_max;        // maximum position holding time 
  INT64             holding_time_avr;        // average position holding time

[fxsaber]

The current version of the tst format does not have the following data

• Time in milliseconds.
• PositionID.
• MagicNumber.

This imposes limitations in usage scenarios.

[fxsaber]

Reproduction of several bugs. Run the Expert Advisor in the Tester on a hedge account.

#include <MT4Orders.mqh> // https://www.mql5.com/ru/code/16006

#define  Ask SymbolInfoDouble(_Symbol, SYMBOL_ASK)

#define  PAUSE 100000

void OnTick()
{
  static bool FirstRun = true;
  
  if (FirstRun)
  {
    OrderSend(_Symbol, OP_BUY, 1, Ask, 0, 0, 0);
    Sleep(PAUSE);
    
    OrderSend(_Symbol, OP_BUY, 2, Ask, 0, 0, 0);
    Sleep(PAUSE);

    if (OrderSelect(0, SELECT_BY_POS))
      OrderClose(OrderTicket(), OrderLots(), OrderClosePrice(), 0);
    Sleep(PAUSE * 2);
    
    if (OrderSelect(0, SELECT_BY_POS))
      OrderClose(OrderTicket(), OrderLots(), OrderClosePrice(), 0);
    Sleep(PAUSE * 2);

    TesterWithdrawal(100);    
    
    FirstRun = false;
  }
}

void OnDeinit( const int )
{
  const int Total = OrdersHistoryTotal();
  
  for (int i = 0; i < Total; i++)
    if (OrderSelect(i, SELECT_BY_POS, MODE_HISTORY))
    {
      OrderPrint();
      
      Print(OrderTicketID()); // MT5-PositionID
    }
}

We get this

2020.01.08 23:59:58   #1 2020.01.01 00:00:00 balance 0.00 0.00000 0.00000 0.00000 2020.01.01 00:00:00 0.00000 0.00 0.00 100000.00 0
2020.01.08 23:59:58   0
2020.01.08 23:59:58   #4 2020.01.02 06:00:00 buy 1.00 EURUSD 1.12137 0.00000 0.00000 2020.01.02 06:03:20 1.12132 -3.56 0.00 -4.46 0
2020.01.08 23:59:58   2
2020.01.08 23:59:58   #5 2020.01.02 06:01:40 buy 2.00 EURUSD 1.12137 0.00000 0.00000 2020.01.02 06:06:40 1.12129 -7.14 0.00 -14.27 0
2020.01.08 23:59:58   3
2020.01.08 23:59:58   #6 2020.01.02 06:10:00 balance 0.00 0.00000 0.00000 0.00000 2020.01.02 06:10:00 0.00000 0.00 0.00 -100.00 withdrawal 0
2020.01.08 23:59:58   0

Next, we read the corresponding tst-file with the script.

#include <fxsaber\SingleTesterCache\SingleTesterCache.mqh> // https://www.mql5.com/ru/code/27611
#include <fxsaber\MultiTester\MTTester.mqh> // https://www.mql5.com/ru/code/26132

void OnStart()
{  
  uchar Bytes2[];
  
  if (MTTESTER::GetLastTstCache(Bytes2) != -1) // If it was possible to read the last cache record of a single run
  {
    const SINGLETESTERCACHE SingleTesterCache(Bytes2); // Drive it into the corresponding object.

    for (int i = 0; i < ArraySize(SingleTesterCache.Positions); i++)
      Print(SingleTesterCache.Positions[i].ToString());
  }
}

It will display data on positions

id = 0
mfe = 0.0
mae = -8.029999999999999
profit = -4.46
lifetime = 00:03:20

id = 0
mfe = 0.0
mae = -21.4
profit = -14.27
lifetime = 00:05:00

id = 0
mfe = 0.0
mae = 0.0
profit = 0.0
lifetime = 00:00:00

If we compare everything in this post, the following bugs are formulated.

• Null id's instead of the correct id's.
• Commission and swap are not taken into account when calculating the profit.
• Withdrawal-trade is mistakenly included in the number of closed trading positions.

[fxsaber]

Instead of set-files I now use tst-files. You can switch between them very quickly, having not only input parameters, but also full backtests.

It is a pity that it is impossible to combine different TSs into a portfolio because of the lack of millisecond data in tst.

I hope the developers will start using the existing fields to their full potential

INT64             TradeDeal::time_create;             // record creation time

INT64             TradeOrder::time_setup;             // time of order reception from the client to the system
INT64             TradeOrder::time_done;              // time of order cancellation

by writing the time value in milliseconds instead of seconds.

In general, in practice to demonstrate all the coolness of using tst do not give very small defects of tst. It should be corrected.

[fxsaber]

It is often necessary to look at the single pass balance graph in more detail.

// Interactive single pass balance graph.

#include <fxsaber\SingleTesterCache\SingleTesterCache.mqh> // https://www.mql5.com/ru/code/27611
#include <fxsaber\MultiTester\MTTester.mqh> // https://www.mql5.com/ru/code/26132

#include <..\Files\Graph.mqh> // https://www.mql5.com/ru/code/18801

#import "shell32.dll"
  int ShellExecuteW( int, string, string, string, string, int );
#import

#define  BASEPATH (::TerminalInfoString(TERMINAL_DATA_PATH) + "\\MQL5\\Files\\")

bool CreateBalanceData( const SINGLETESTERCACHE &SingleTesterCache, const string FileName = "exdat.txt" )
{
  const int handle = FileOpen(FileName, FILE_WRITE | FILE_TXT | FILE_ANSI);
  const bool Res = (handle != INVALID_HANDLE);

  if (Res)
  {
    FileWriteString(handle, "var dat1=[\n");
    
    for (uint i = 0; i < SingleTesterCache.Header.equities_total; i++)
      FileWriteString(handle, "[" + (string)((long)SingleTesterCache.TradeState[i].time * 1000) + "," + ::DoubleToString(SingleTesterCache.TradeState[i].balance, 2) + "],\n");

    FileWriteString(handle, "];\n");
    FileWriteString(handle, "var T1=dat1[0][0];\n");
    FileWriteString(handle, "var T2=dat1[dat1.length-1][0];\n");
    FileWriteString(handle, "var nTrades=dat1.length;\n");
    FileWriteString(handle, "var Balance=" + ::DoubleToString(SingleTesterCache.TradeState[SingleTesterCache.Header.equities_total - 1].balance, 2) + ";\n");
    FileWriteString(handle, "var Currency=\"" + (SingleTesterCache.Header.trade_pips ? "Pips" : SingleTesterCache.Header.trade_currency[]) + "\";\n");

    FileClose(handle);
  }

  return(Res);
}

void OnStart()
{  
  uchar Bytes2[];
  
  if (MTTESTER::GetLastTstCache(Bytes2) != -1) // If it was possible to read the last cache record of a single run
  {
    const SINGLETESTERCACHE SingleTesterCache(Bytes2); // Drive it into the corresponding object.
    
    const string FileName = "Report.htm";
    uchar Array[];    
    
    if ((StringToCharArray(StrMQH, Array) > 0) && FileSave(FileName, Array) && CreateBalanceData(SingleTesterCache))
      ShellExecuteW(0, "Open", BASEPATH + FileName, NULL, NULL, 3);      
  }
}

If we replace

TradeState[i].balance -> TradeState[i].equity

it's an equity graph.

[fxsaber]

The volume of trades/orders is incorrectly written in tst. It is always calculated as if SYMBOL_TRADE_CONTRACT_SIZE = 100 000.

If a different value is set, it has no effect on the volume value in tst.

[fxsaber]

fxsaber:

in practice to demonstrate all the coolness of using tst in practice is not given at all by small shortcomings of tst. It should be corrected.

Thefirst swallow.

[Edgar Akhmadeev]

fxsaber:

First swallow.

https://www.mql5.com/ru/blogs/post/732868#comment_14614000

[fxsaber]

Edgar Akhmadeev:

https://www.mql5.com/ru/blogs/post/732868#comment_14614000

Answered there.

[Edgar Akhmadeev]

I don't know how it happens on blogs, is the appearance of a new comment (if it's not a reply) signalled somehow? Or is it better to post in one of the forum threads where new comments are visible?

Why didn't they publish it in the KB? It would be more convenient.

So where to post for prompt response?