Libraries: TesterBenchmark - page 2

New comment
 

The idea is good, but the implementation is not right. Answered in a private message:

fxsaber

Good afternoon!

Could you adapt the trading logic to your trading library to measure its performance?

Forum on trading, automated trading systems and testing trading strategies.

Libraries: TesterBenchmark

fxsaber, 2017.08.15 19:31

The places were distributed as follows

  1. Pure MQL5 - 100% performance.
  2. MT4Orders.mqh - ~95% performance.
  3. SB Trade\Trade.mqh - ~84% performance.
SB started to lag less.


ZЫ I wonder how much other trading libraries show...

Thanks!

I don't understand the performance of what? The library? It doesn't work without a strategy. You need a strategy. Then we need to measure the performance of the strategy written with the help of the library. Let's say we measured it, got the result. What should we compare it to? We need to compare it with the same strategy written in pure MQL. Where to get two strategies? Suppose we have such two strategies. But where is the guarantee that they are identical in terms of trading entries/exits with each other? Suppose we have two strategies that are guaranteed to be identical to each other, we measure their speeds. The velocities differ only slightly. Does this mean that the engine is efficient? No, it doesn't, because the strategy doesn't use all the engine/MQL capabilities. Suddenly it turns out that if you request a long history of quotes in another strategy, the engine starts working many times slower. Does this mean that the engine works many times slower than pure MQL? No, it does not mean that this particular operation is slower, but let's say that if you optimise the classes that provide data, in new versions it will turn out that the engine will work faster than the system CopyXXX on the same operation. That's why the main thing is not clear in this whole story: what we are and what we are comparing to.

 
Vasiliy Sokolov:

The idea is good, but the implementation is not right. Answered in a private message:

Here is the code of the test EA.

It contains the same simple trading logic, but in three ways: pure MQL5, Trade.mqh and MT4Orders.mqh

This allows us to measure the performance of each variant with absolutely identical trading results.

So far it turns out that Trade.mqh is 15% behind. But the performance measurement approach is universal - you can repeat the trading logic on any trading API. Here I suggest you to write it on your cross-platform bible and measure the speed by comparing it with pure MQL5.

Thanks to such measurements, I was able to find bottlenecks in my bible and eradicate them. And when optimising, this sometimes translates into hours saved. In general, I think it is useful.


In your advisor, you do only
#include <TesterBenchmark.mqh>

and you don't need anything else. Then select Optimisation and run it. You will see the performance in the logs.

 

In your code of the test Expert Advisor, at each tick, a complete search of deals on history is made, taking into account their total lot and profit, and the volume of the current deal is calculated:

if (HistorySelect(0, TimeCurrent()))
    {
      const int Total = HistoryDealsTotal() - 1;
      double SumProfit = 0;
      double SumLots = 0;
      for (int i = Total, Count = 0; (i >= 0) && (Count < AmountLastDeals); i--)
      {
        const ulong Ticket = HistoryDealGetTicket(i);

        if ((ENUM_DEAL_ENTRY)HistoryDealGetInteger(Ticket, DEAL_ENTRY) == DEAL_ENTRY_OUT)
        {
          SumProfit += HistoryDealGetDouble(Ticket, DEAL_PROFIT) * (AmountLastDeals - Count);
          SumLots += HistoryDealGetDouble(Ticket, DEAL_VOLUME) * (AmountLastDeals - Count);

          Count++;
        }
      }
      SumLots /= (AmountLastDeals * (1 + AmountLastDeals)) >> 1;
      ...
   }

Your Expert Advisor does nothing but load on HistorySelect. Therefore, it cannot be a benchmark for measuring the performance of anything but HistorySelect itself. Therefore, your statements that I quote:"MT5 is 4.4 times faster than MT4!" should at least be corrected to HistorySelect in MT5 is 4.4 times faster than a similar, internal procedure in MT4.

As for my engine CStrategy, it currently does not allow to work with trades at all, but only with positions, so the example as it is written now cannot be used in CStrategy.

P.S. I will note that in your example the speed of work is equal to O(n). And the only difference between MT4 and MT5 in this case is the speed of HistorySelect/HistoryOrderSelect. Naturally, CStrategy in tasks with complexity O(n) will use basic calls, in this case HistorySelect and naturally, it will be a bit slower than calling HistorySelect directly. But when you need to call, say, HistoryOrderSelect(ticket) - here everything can be much more interesting and in some tasks CStrategy will break the standard access to the environment, due to access by dictionary and storing data in the memory of the programme.

 
Vasiliy Sokolov:

In your code of the test Expert Advisor, at each tick, a complete search of deals on history is made, taking into account their total lot and profit, and the volume of the current deal is calculated:

Your Expert Advisor does nothing but load on HistorySelect. Therefore, it cannot be a benchmark for measuring the performance of anything but HistorySelect itself. Therefore, your statements that I quote:"MT5 is 4.4 times faster than MT4!" should at least be corrected to HistorySelect in MT5 is 4.4 times faster than a similar, internal procedure in MT4.

You have jumped to conclusions. The given piece is executed only on opening a position once every Interval-seconds. That is, very rarely.

As for my CStrategy engine, it currently does not allow working with trades at all, but only with positions, so the example as it is written now cannot be used in CStrategy.

Here is the MQL4 code of the OnTick test EA, which also works only with positions.

if (!OrderSelect(0, SELECT_BY_POS))
{
  const int Total = OrdersHistoryTotal() - 1;

  double SumProfit = 0;
  double SumLots = 0;

  for (int i = Total, Count = 0; (i >= 0) && (Count < AmountLastDeals); i--)
    if (OrderSelect(i, SELECT_BY_POS, MODE_HISTORY) && (OrderType() <= OP_SELL))
    {
      SumProfit += OrderProfit() * (AmountLastDeals - Count);
      SumLots += OrderLots() * (AmountLastDeals - Count);

      Count++;
    }

  SumLots /= (AmountLastDeals * (1 + AmountLastDeals)) >> 1;

  const int Type = (Total >= 0) && OrderSelect(Total, SELECT_BY_POS, MODE_HISTORY) && (OrderType() == OP_BUY) ? OP_SELL : OP_BUY;

  OrderSend(_Symbol, Type, (SumProfit >= 0) ? Lots : CorrectLot(SumLots),
            SymbolInfoDouble(_Symbol, (Type == OP_BUY) ? SYMBOL_ASK : SYMBOL_BID), 0, 0, 0); // , NULL, 0, 0, 0, INT_MIN);
}
else if (TimeCurrent() - OrderOpenTime() >= Interval)
  OrderClose(OrderTicket(), OrderLots(), OrderClosePrice(), 0);

It is not difficult to rewrite such simple logic on your API, of course. But if you have a suggestion for a better EA to analyse the performance of the trading API (just trading, not time series, etc.), I will be glad to see your version of the trading logic. I can quickly rewrite it for pure MQL5, MQL4 and Trade.mqh.


ZY When starting a test advisor choose balance = 1e7. To always have enough funds to open a position.

 
fxsaber:

You have jumped to conclusions. The given piece is executed only at position opening once every Interval-seconds. That is, very rarely.

Here is the MQL4 code of OnTick of the test Expert Advisor, which also works only with positions.

It is not difficult to rewrite such simple logic on your API, of course. But if you have a suggestion for a better EA to analyse the performance of the trading API (just trading, not time series, etc.), I will be glad to see your version of the trading logic. I can quickly rewrite it for pure MQL5, MQL4 and Trade.mqh.

Take a close look at your code and tell me what it measures

if (!OrderSelect(0, SELECT_BY_POS))
{
  const int Total = OrdersHistoryTotal() - 1;

  double SumProfit = 0;
  double SumLots = 0;

  for (int i = Total, Count = 0; (i >= 0) && (Count < AmountLastDeals); i--)
    if (OrderSelect(i, SELECT_BY_POS, MODE_HISTORY) && (OrderType() <= OP_SELL))
    {
      SumProfit += OrderProfit() * (AmountLastDeals - Count);
      SumLots += OrderLots() * (AmountLastDeals - Count);

      Count++;
    }

  SumLots /= (AmountLastDeals * (1 + AmountLastDeals)) >> 1;

  const int Type = (Total >= 0) && OrderSelect(Total, SELECT_BY_POS, MODE_HISTORY) && (OrderType() == OP_BUY) ? OP_SELL : OP_BUY;

  OrderSend(_Symbol, Type, (SumProfit >= 0) ? Lots : CorrectLot(SumLots),
            SymbolInfoDouble(_Symbol, (Type == OP_BUY) ? SYMBOL_ASK : SYMBOL_BID), 0, 0, 0); // , NULL, 0, 0, 0, INT_MIN);
}
else if (TimeCurrent() - OrderOpenTime() >= Interval)
  OrderClose(OrderTicket(), OrderLots(), OrderClosePrice(), 0);

There are two bottlenecks here: the for loop and the order sending itself. Do you want to compare where the for loop works faster? In CTrade or in pure MQL? Where is it faster to send an order? In CTrade or OrderSend? Of course in pure OrderSend is faster, because in CTrade a lot of checks are done before the order is sent. But what conclusions should be drawn from this?

 
Vasiliy Sokolov:

Take a close look at your code and tell me what it measures

There are two bottlenecks here: the for loop and the actual sending of the order. Do you want to compare where the for loop works faster?

For some reason you don't see it

fxsaber:

The given piece is executed only on opening a position once every Interval-seconds. I.e. very rarely.


Vasiliy Sokolov:

Where is it faster to send an order? In CTrade or OrderSend? Of course in pure OrderSend it is faster, because in CTrade a lot of checks are done before the order is sent. But what are the implications of this?

CTrade does much less checks than the same MT4Orders. The conclusions are simple now - SB lags behind pure MQL5 by 16% and can be accelerated absolutely accurately. And anyone who uses CTrade loses extra money in the Cloud during Optimisation and time for the Optimisation itself.

 

You still do not understand what your code is doing. I ran the profiler on purpose and within an hour I made 1800 trades with 1 second intervals. 99,66% of OnTick execution time is taken up by two OrderSend functions:

What are you going to optimise? The remaining 0.34% is meaningless. You cannot optimise OrderSend.

Okay. The most interesting part. Put #include <Trade\Trade.mqh> and see what has changed when using the CTrade library:


It now takes 100% of the time to call Trade.Buy/Sell and Trade.PositionClose. We go to Trade.PositionClose and ta-da-mmmm:

I.e. all 99.97% of the time Trade.PositionClose takes the same OrderSend. Similarly with Trade.Buy and Trade.Sell.

So what are you going to optimise in SB? Where is the subject of the test? What are you testing in general?

(For those who do not understand the profiling figures, I will explain: 99.97% of the programme execution time is taken by direct order sending via OrderSend, the remaining 0.03% is taken by CTrade binding with necessary checks before order sending. That is, using a competent OOP introduces such negligible delays that they can be neglected and considered to be equal to zero. It is not 0.03%, but even less, because most of the time of this 0.03% is taken by mandatory checks, which should be implemented in a competent way, whether in procedural code or somewhere else).

 
Vasiliy Sokolov:

I specifically ran the profiler and within an hour I was hitting 1800 trades at 1 second intervals.

What does online have to do with it? It's about the tester.

 
Andrey Khatimlianskii:

What does online have to do with it? It's about the tester.

If we are talking about MT5 and its strategy tester, the same 99% of time (well, maybe a little less) is spent there on recreating the trading environment (hence such slow runs compared to MT4). And there is no reason to think that if OrderSend in real life takes 99.9% of time, then in the strategy tester it will suddenly work two orders of magnitude faster.
 
Vasiliy Sokolov:
If we are talking about MT5 and its strategy tester, the same 99% of time (well, maybe a little less) is spent there on recreating the trading environment (hence such slow runs compared to MT4). And there is no reason to think that if OrderSend in real life takes 99.9% of the time, it will suddenly work two orders of magnitude faster in the strategy tester.

Vasily, are you seriously talking about the commensurability of OrderSend execution speed in the tester and in real time?

Online it is the slowest because it sends information to the server and waits for a response, while in the tester (if you don't include the delay, but it was out of the question) it sends and waits instantly.

The task of this library is to measure the speed of the tester (in terms of trading functions).

1234
New comment