MT5 and speed in action - page 9

New comment
 
fxsaber:

Turns out to be a frequent occurrence. Trading functions have not been called.

SymbolInfoTick is not a bad lag sometimes. HFT may be very experienced with such unexpected lags.

Please ask the Developers to find the reasons. In the meantime, it is obvious that in battle EAs their profiler is a must.

What will the test show on the "empty" terminal?

void OnStart()
  {
   MqlTick Tick;
//---
   ulong start,end,max_time=0,avr_time=0;
   int   count=100000;
   for(int i=0; i<count; i++)
     {
      start=GetMicrosecondCount();
      SymbolInfoTick(_Symbol, Tick);
      end=GetMicrosecondCount()-start;
      //--- >1 ms
      if(end>1000)
         Print(" > 1 ms for one SymbolInfoTick: ",DoubleToString(end/1000.0,2)," ms");
      //---
      if(end>max_time)
         max_time=end;
      avr_time+=end;
     }
   Print("SymbolInfoTick max time: ",DoubleToString(max_time/1000.0,3)," ms; avr time: ",DoubleToString(avr_time/1000.0/count,3)," ms; ",count," iterations");
//---
   start=GetMicrosecondCount();
   for(int i=0; i<count; i++)
      SymbolInfoTick(_Symbol, Tick);
   end=GetMicrosecondCount()-start;
   Print(count," SymbolInfoTick = ",DoubleToString(end/1000.0,2)," ms");
  }

It should be something like this:

2020.06.04 11:02:30.123 TestSymbolInfoTick (EURUSD,H1)  SymbolInfoTick max time: 0.138 ms; avr time: 0.000 ms; 100000 iterations
2020.06.04 11:02:30.138 TestSymbolInfoTick (EURUSD,H1)  100000 SymbolInfoTick = 14.85 ms
2020.06.04 11:02:31.433 TestSymbolInfoTick (EURUSD,H1)  SymbolInfoTick max time: 0.051 ms; avr time: 0.000 ms; 100000 iterations
2020.06.04 11:02:31.448 TestSymbolInfoTick (EURUSD,H1)  100000 SymbolInfoTick = 15.17 ms
2020.06.04 11:02:33.064 TestSymbolInfoTick (EURUSD,H1)  SymbolInfoTick max time: 0.035 ms; avr time: 0.000 ms; 100000 iterations
2020.06.04 11:02:33.079 TestSymbolInfoTick (EURUSD,H1)  100000 SymbolInfoTick = 15.12 ms

Unless you tell us in detail what you are doing, how exactly you are putting a load on the terminal, it will be difficult for us to find the reasons.

 
Anton:

What will the test show on an "empty" terminal?

It should be something like this:

        SymbolInfoTick max time: 0.034 ms; avr time: 0.000 ms; 100000 iterations
        100000 SymbolInfoTick = 8.56 ms
        SymbolInfoTick max time: 0.047 ms; avr time: 0.000 ms; 100000 iterations
        100000 SymbolInfoTick = 9.04 ms
        SymbolInfoTick max time: 0.045 ms; avr time: 0.000 ms; 100000 iterations
        100000 SymbolInfoTick = 9.02 ms

Unless you tell us in detail what you are doing, exactly how you are putting the load on the terminal, it will be hard for us to find the reasons.

100K iterations is not an indicator. Since the function does not always slow down, but sometimes.

In fact I need to disable chunks of the combat EA until the braking stops. Then I can provide the code. We have to wait.

 
fxsaber:

In fact, I need to disable chunks of the combat advisor until the brakes stop. Then I can provide the code. I have to wait.

Run this EA on a few characters to get a quick result.

#include <fxsaber\Benchmark.mqh> // https://c.mql5.com/3/321/Benchmark.mqh

void OnTick()
{
  MqlTick Tick[1];
  
  if (_B(CopyTicks(_Symbol, Tick, COPY_TICKS_ALL, 0, 1), 1)) // Не знаю, влияет это или нет.
    _B(SymbolInfoTick(_Symbol, Tick[0]), 1);
}


Got it in five minutes.

        Alert: Time[Test6.mq5 8: SymbolInfoTick(_Symbol,Tick[0])] = 9 ms.
        Alert: Time[Test6.mq5 8: SymbolInfoTick(_Symbol,Tick[0])] = 3 ms.
        Alert: Time[Test6.mq5 8: SymbolInfoTick(_Symbol,Tick[0])] = 4 ms.
        Alert: Time[Test6.mq5 8: SymbolInfoTick(_Symbol,Tick[0])] = 1 ms.
        Alert: Time[Test6.mq5 8: SymbolInfoTick(_Symbol,Tick[0])] = 2 ms.
        Alert: Time[Test6.mq5 8: SymbolInfoTick(_Symbol,Tick[0])] = 5 ms.
        Alert: Time[Test6.mq5 8: SymbolInfoTick(_Symbol,Tick[0])] = 2 ms.
        Alert: Time[Test6.mq5 8: SymbolInfoTick(_Symbol,Tick[0])] = 1 ms.
        Alert: Time[Test6.mq5 8: SymbolInfoTick(_Symbol,Tick[0])] = 16 ms.
        Alert: Time[Test6.mq5 8: SymbolInfoTick(_Symbol,Tick[0])] = 16 ms.


Looks like it's enough to leave only this (without CopyTicks) in the EA.

    _B(SymbolInfoTick(_Symbol, Tick[0]), 1);
 
fxsaber:

100K iterations is not an indicator. As the function does not always slow down, but sometimes.

I propose to change the concept of defining the fastness of a function.

A function is fast if there are no peaks in its execution time.


As was shown above, even simple functions have such peaks. Sometimes very large ones. I have no idea what this has to do with. But it is obvious that all trading-critical functions should be checked for the presence of peaks using the method suggested above. I.e. we run and monitor peaks greater than a millisecond for a couple of hours.


It is necessary to achieve that there are no peaks at least on an empty Terminal. Fast 100K iterations turn out to be nothing.

 
fxsaber:

I suggest changing the concept of defining the fastness of a function.

A function is fast if there are no peaks in its duration.


As shown above, even simple functions have such peaks. Sometimes very large ones. I have no idea what this has to do with it. But it is obvious that all trading-critical functions should be checked for the presence of peaks using the method suggested above. I.e. we run and monitor peaks greater than a millisecond for a couple of hours.


It is necessary to achieve that there are no peaks at least on an empty Terminal. Fast 100K iterations turn out to be nothing.

Sometimes it happens that the timer shows a cumulative amount of time if another task is running. For example, it may happen when working with canvas - when displaying function sets the task to display without creating an image and comes back. After that any other function is executed sequentially, for example the same comment, however the process of canvas mapping is started in CPU language and only after that kanvas is displayed. When measuring the timing you can see that the comment takes a very long time to be output, but the kanvas display function runs in 0 ms.

 
fxsaber:

run and monitor peaks greater than a millisecond for a couple of hours.

We need to ensure that there are no peaks at least on an empty Terminal. Fast 100K iterations turned out to be worthless.

I drafted such an Expert Advisor for monitoring.

// Мониторинг длительных пиков выполнения важных функций для торговли.
#include <fxsaber\Benchmark.mqh> // https://c.mql5.com/3/321/Benchmark.mqh

input int inCycle = 10;    // Циклов проверки в одном OnTick
input int inAlertTime = 1; // Нижний порог в миллисекундах

#define _B2(A) _B(A, AlertTime)

void Check( const string Symb, const int AlertTime = 1 )
{
  MqlTick Tick;
  
  if (_B2(SymbolInfoTick(Symb, Tick)))
  {
    MqlTick Ticks[];
    
    _B2(CopyTicks(Symb, Ticks, COPY_TICKS_ALL, 0, 1));
    _B2(CopyTicks(Symb, Ticks, COPY_TICKS_ALL, Tick.time_msc));
    _B2(CopyTicksRange(Symb, Ticks, COPY_TICKS_ALL, Tick.time_msc));

    _B2(HistorySelect(Tick.time, INT_MAX));
    
    _B2(HistoryDealsTotal());
    _B2(HistoryDealGetTicket(0));
    _B2(HistoryDealGetInteger(0, DEAL_MAGIC));
    _B2(HistoryDealGetDouble(0, DEAL_PRICE));
    _B2(HistoryDealSelect(0));

    _B2(HistoryOrdersTotal());
    _B2(HistoryOrderGetTicket(0));
    _B2(HistoryOrderGetInteger(0, ORDER_MAGIC));
    _B2(HistoryOrderGetDouble(0, ORDER_PRICE_CURRENT));
    _B2(HistoryOrderSelect(0));
    
    _B2(GetLastError());
    _B2(IsStopped());
    
    _B2(SymbolInfoDouble(Symb, SYMBOL_ASK));
    _B2(SymbolInfoDouble(Symb, SYMBOL_TRADE_TICK_VALUE));
    _B2(SymbolInfoDouble(Symb, SYMBOL_POINT));
    _B2(SymbolInfoInteger(Symb, SYMBOL_DIGITS));

    _B2(TimeCurrent());
    _B2(TimeLocal());
    _B2(TimeTradeServer());
    
    _B2(OrdersTotal());
    _B2(OrderSelect(0));
    _B2(OrderGetDouble(ORDER_PRICE_CURRENT));
    _B2(OrderGetInteger(ORDER_MAGIC));
    _B2(OrderGetString(ORDER_SYMBOL));
    
    _B2(PositionsTotal());
    _B2(PositionSelect(Symb));
    _B2(PositionSelectByTicket(0));
    _B2(PositionGetDouble(POSITION_PRICE_CURRENT));
    _B2(PositionGetInteger(POSITION_MAGIC));
    _B2(PositionGetString(POSITION_SYMBOL));
    
    _B2(AccountInfoDouble(ACCOUNT_EQUITY));
    _B2(AccountInfoInteger(ACCOUNT_MARGIN_MODE));
    
    MqlTradeRequest Request = {0};
    MqlTradeCheckResult CheckResult;

    _B2(OrderCheck(Request, CheckResult));
    
    _B2(MQLInfoInteger(MQL_TRADE_ALLOWED));
    _B2(AccountInfoInteger(ACCOUNT_TRADE_EXPERT));
    _B2(AccountInfoInteger(ACCOUNT_TRADE_ALLOWED));
    _B2(TerminalInfoInteger(TERMINAL_TRADE_ALLOWED));
    
    _B2(SymbolsTotal(true));
    _B2(SymbolName(0, true));
    _B2(Symbol());
    
    _B2(GlobalVariableCheck(NULL));
    _B2(GlobalVariableGet(NULL));
    
    _B2(ResourceFree(NULL));
  }
}

void OnTick()
{
  for (int i = 0; i < inCycle; i++)
    Check(_Symbol, inAlertTime);
}


I got the result in five minutes of monitoring.

        Alert: Time[Test6.mq5 18: HistorySelect(Tick.time,INT_MAX)] = 21 ms.
        Alert: Time[Test6.mq5 24: HistoryDealSelect(0)] = 4 ms.
        Alert: Time[Test6.mq5 10: SymbolInfoTick(Symb,Tick)] = 24 ms.
        Alert: Time[Test6.mq5 14: CopyTicks(Symb,Ticks,COPY_TICKS_ALL,0,1)] = 4 ms.
        Alert: Time[Test6.mq5 30: HistoryOrderSelect(0)] = 3 ms.
        Alert: Time[Test6.mq5 35: SymbolInfoDouble(Symb,SYMBOL_ASK)] = 2 ms.


SZZY With such value of input parameter there are much less alerts.

input int inAlertTime = 10; // Нижний порог в миллисекундах


But the result is more significant too.

        Alert: Time[Test6.mq5 21: HistorySelect(Tick.time,INT_MAX)] = 19 ms.
        Alert: Time[Test6.mq5 21: HistorySelect(Tick.time,INT_MAX)] = 10 ms.
        Alert: Time[Test6.mq5 38: SymbolInfoDouble(Symb,SYMBOL_ASK)] = 10 ms.
        Alert: Time[Test6.mq5 13: SymbolInfoTick(Symb,Tick)] = 10 ms.
        Alert: Time[Test6.mq5 38: SymbolInfoDouble(Symb,SYMBOL_ASK)] = 10 ms.
        Alert: Time[Test6.mq5 17: CopyTicks(Symb,Ticks,COPY_TICKS_ALL,0,1)] = 148 ms.
        Alert: Time[Test6.mq5 74: SymbolName(0,true)] = 11 ms.
 

Finally, with me modifying a bunch of orders sometimes takes 3-10 seconds per order. After that it takes a long time of 0.1 seconds again.

Pulled up the server logs - it's instantaneous there.


It is very unpleasant on a battle Expert Advisor.

2020.06.04 15:24:48.771 Alert: Time[NewTicks.mqh 31: ::SymbolInfoTick(_Symbol,Tick)] = 61 ms.
2020.06.04 15:25:21.729 Alert: Time[NewTicks.mqh 31: ::SymbolInfoTick(_Symbol,Tick)] = 29 ms.
2020.06.04 15:27:57.842 Alert: Time[NewTicks.mqh 31: ::SymbolInfoTick(_Symbol,Tick)] = 142 ms.


Some fantastic values.

 
fxsaber:

Finally, with me modifying a bunch of orders sometimes takes 3-10 seconds per order. After that it takes a long time of 0.1 seconds again.

Raised server logs - there instantly.

The situation was repeated on another trading server.

The terminal modified the open position take 2.5 seconds. On the server - 2 milliseconds.

Most likely, this is also the source of problems with FORTS-execution.

ФОРТС. Вопросы по исполнению
ФОРТС. Вопросы по исполнению
  • 2020.03.30
  • www.mql5.com
С большими проблемами удалось это сделать (начальник отдела по работе с профессиональными клиентами ДЦ Открытие Евгений Сергеевич,.
 

Retransmits.

Forum on trading, automated trading systems and trading strategies testing

MetaTrader 5 build 1700 platform beta: Projects in MetaEditor and synthetic tools

Renat Fatkhullin, 2017.12.14 12:47

This is an indicator of the quality of communication. The percentage of retransmitted network packets in the TCP/IP protocol.

It is calculated globally at the network interface level for all applications across the entire operating system. When you suspect slowness and problems, look at this metric. Critical when the broker's server is very far away. For example for Asian traders and a broker in Europe.

Already at 3% retransmit rate you can tell you can't trade. The extreme level of retransmits is given by bad wifi.

Forum on trading, automated trading systems and trading strategies testing

New MetaTrader 5 build 2360: Expanded Integration with SQLite

Renat Fatkhullin, 2020.04.06 12:33

The norm should be less than 1%. And already 3% network loss kills low latency services.

For example, our retransmits are 0.68 - 0.75% with obviously more users (we have 17k online on MetaQuotes-Demo). And we serve the whole world, not Moscow/Russia.

Forum on trading, automated trading systems and trading strategy testing

Bugs, bugs, questions

Renat Fatkhullin, 2017.12.17 23:03

These are statistics of the network interface of the whole computer, where Metatrader is just one of the users. It is not necessarily related to the trading server.

General stats can easily be corrupted by a web browser after failing to access some glitchy and distant site. It is also possible for a local wifi to catch a network conflict and get tens of percent retransmits at random points in time.

In case of 20% retransmits there will be no connection to the trade server and the reconnects will be constant and endless. Terminal has constant connection and even 3-5% retransmits will be fatal for it to maintain long connections.

Forum on trading, automated trading systems and trading strategies testing

Bugs, bugs, questions

Renat Fatkhullin, 2017.12.18 11:36

Keep in mind that this is a technical characteristic of your local TCP/IP stack, reported by the OS, not an indicator of the quality of a particular connection to the trading servers. It includes all network activity, including system/phone activity.


The connection of a trading cluster is known to be of high quality and we log a lot of parameters (this is a standard platform functionality), collecting one-minute snapshots and subsequent analysis.

Forum on trading, automated trading systems and trading strategies testing

Bugs, bugs, questions

Renat Fatkhullin, 2017.12.18 00:13

Checked.

None of the nodes in our demo cluster, including Asia, had any restarts or retransmit level increase for the whole day (and on other days too). Everything is between 0.5% and 1.5% normal.


I seem to have a lot.

It's midnight now, quoters are rarely updated. Retransmits are increasing before my eyes. I want to put Alert on VPS to a high value > 1% for low latency trading. But with such huge values this idea becomes pointless.


What can I recommend? Do tracert to trade server? Some kind of monitoring program? In general, how to make sure that MT5 is ready for low latency?


ZS As soon as the quotes start moving faster, the index drops many times over.

 
fxsaber:

Retransmitters.


I seem to have a lot of it.

5-6am:

Home (optic, ETH to router, cable to computer) - 8-19%, ping 60-70

VPS in Netherlands (momentarily 1 MT5 with 9 currencies/11 charts) - 1.2-1.6%, ping 3.7

12345678910111213141516...94
New comment
Reason: