MT5 and speed in action - page 78

New comment
 
Valeriy Yastremskiy:

Correctly understand that the terminal memory is stacked with the terminal program memory of mcl, tix and others.

No. Simply CopyTicks stores cache of requested ticks for 10 seconds. For example, requested 3 Gb of memory, Terminal will store this 3 Gb more in cache. Total Terminal will consume 6 GB. If you do ArrayFree and request 3 Gb for another symbol, the terminal will consume 9 Gb. And so on.

 
Valeriy Yastremskiy:

Correctly understand that the memory of the terminal is added to the memory of the terminal programs mcl, tix and others.

Of course.
 
We need to make a script that creates a custom symbol based on tick history of several symbols by formula 
MqlTick Formula( const MqlTick &Symbol1_Tick
                 const MqlTick &Symbol2_Tick,
                 const MqlTick &Symbol3_Tick,
                 const MqlTick &Symbol4_Tick,
                 const MqlTick &Symbol5_Tick );

It would seem that 100 MB of RAM should be enough for this task, even for a billion ticks. But in MT5 you can't solve this task via CopyTicks.

Here's the crutch:

  1. CopyTicks is called for each symbol separately (and necessarily with waiting for cache release after each call), writing tick history to its corresponding files via FileSave.
  2. Then comes reading ticks from these files and calling Formula for them.

Yes, it's a creepy crutch, but there's no other options. So, you can't work with CopyTicks directly. It's necessary to use file archives of the ticks.


The biggest memory consumption will be in p.1. even with condition of expectation of cache releasing after each call. In this case p.2. will run for free!

 

I am trading on 8 pairs at once, with several Expert Advisors on each pair. And although in terms of resources it looks good, memory usage is not more than 25%, CPU is not loaded more than 10%, lags are noticeable, like a new chart opens for a few seconds, as well as in general trading. Maybe there are some best practices, at least in the abstract, how to pack it all and make it work faster? I know that you are virtualizing several Expert Advisors into one. What are the pitfalls? How has the sending of trade orders been implemented? What should I pay attention to?

P.S. I myself use synchronization through Virtual and trade through MT4Orders, 1 EA per chart.

 
fxsaber:

Likewise. Take your VPS. Market screeners can't work on it.

ZS It would be good to get rid of the hiccups that have been happening for months. Run this script on a machine with infinite RAM. For example, I can't upload ticks from June 1st just one character at a time. It just hangs CopyTicks with zero resource consumption.

Take a dump of the terminal when it "hangs". Let's see what the cause is.

 
traveller00:

I am trading on 8 pairs at once, with several Expert Advisors on each pair. And although in terms of resources it looks good, memory usage is not more than 25%, CPU is not loaded more than 10%, lags are noticeable, like a new chart opens for a few seconds, as well as in general trading. Maybe there are some best practices, at least in the abstract, how to pack it all and make it work faster?

Use a fresh MT4Orders (where the history handling is accelerated after this branch was created) and snapshot the current environment: orders and positions. I have everything flying.

I know that you are virtualizing several Expert Advisors into one. What are the pitfalls? How has the sending of trade orders been implemented? What should I pay attention to?

I am using synchronous OrderSend, but I am disabling the MT5 checks.

Forum on trading, automated trading systems & strategy testing

Libraries: MT4Orders

fxsaber, 2020.09.29 08:45

With this line.

MT4ORDERS::OrderSend_MaxPause = 0; // Отключение проверки корректности работы MT5-OrderSend.

can be used to disable it all. May be useful in case of sluggish MT5 trading history, because MT4Orders checks correctness (and even corrects) of MT5-OrderSend sometimes through reference to this history.

I do not recommend doing this.


Each set (set of input parameters) is given its own Virtual. The synchronizer goes through all the Virtuals and does synchronization for each one. It needs to do this in a loop, not from the first Virtual.

After any OrderSend call (inside the synchronizer) a snapshot is necessarily made and fresh ticks are added (to all Virtuals) that came during OrderSend execution time. I.e. after any hypothetical pause we make everything fresh.

Fresh ticks are taken only via CopyTicks each time. No SymbolInfoTick to roll through. Make sure that if CopyTicks_LastTick.time_msc < SymbolInfoTick.time_msc (it happens often, even if calls are one after another (in any sequence), then the synchronizer is not enabled. Otherwise, you may encounter that the real time limit is executed, but the virtual time limit is not. And there will be problems with synchronization.

I make snapshots through VIRTUAL::Snapshot. In addition to the obvious speed, it also allows you to separate your symbol from the rest - only your symbol gets there. This gives even more speed. Plus, not only history is disabled in the snapshot,

#define  VIRTUAL_SNAPSHOT_WITHOUT_HISTORY // Отказ от снепшота истории для повышения производительности

but also fields (marked below), which require access to history.

#define  MACROS(A) this.##A = ::Order##A();

  bool ORDERS::Copy( const bool WithoutHistory = false )
  {
    MACROS(CloseTimeMsc)

    if (WithoutHistory && !this.CloseTimeMsc) // Для исторических ордеров оставляем все без изменений.
    {
      const string Str = NULL;
      this.comment = Str;

      this.Commission = 0;
      this.OpenPriceRequest = this.OpenPrice;
    }
    else // В MT4Orders требуется обращение к истории.
    {
      const string Str = ::OrderComment();
      this.comment = Str;

      MACROS(Commission)
      MACROS(OpenPriceRequest)
    }

Previously I also checked if LastDeal.time_msc was not larger than LastTick.time_msc. If this condition was not fulfilled, I refused synchronization for obvious reasons. But such checking consumes a lot of resources (it works with history), so I refused.


Trade On-function - OnTick.


I have probably listed the main ones.

 

Synchronous, I take it, not onlyOrderSend, but all trade orders including modification, deletion etc.?


SymbolInfoTick is not thrown through, because the order of ticks can mess up? And CopyTicks are exactly in the right order.


It turns out that SymbolInfoTick is needed only to check the time and that's all? All the trades, etc. are corrected only byCopyTicks?


Is there any sense in trying to pack several EAs in 1 on 1 chart? Nothing impossible, but trying to understand if it's worth the hassle and rewrite or the profit will be minimal?

 
traveller00:

Does it make sense to try to pack several EAs into 1 on 1 chart? Nothing impossible, but trying to figure out if it's worth the hassle and rewrite or will the profit be minimal?

Where would the profit come from? Everyone will be waiting for any of them when trading.

And there is just less paralleling.

Unless you can keep one cache of orders for all, but the benefit is questionable...

 
Anton:

Take a dump of the terminal when it "hangs up". See what's causing it.

Isn't the code reproducing the problem enough for you? Or is it not reproducing?

 
traveller00:

Synchronous, I take it, not onlyOrderSend, but all trade orders including modification, deletion, etc.?

I meant MT5-OrderSend.

SymbolInfoTick is not passed, because the order of ticks can be spoiled? And CopyTicks goes exactly all in correct order.

Because there will be holes.

So, SymbolInfoTick is required only for checking of time, that's all?

Yes.

All the trade, etc. will go only byCopyTicks?

Only.

Does it make sense to try to pack several EAs in 1 on 1 chart? Nothing impossible, but trying to figure out if it's worth the hassle and rewrite or the profit will be minimal?

This is a huge help to me.


Every single pass (in single test mode) at the end writes its insteps to the same file. So after looking at some passes after Optimisation I get a file with the data of those passes. Then I filter them in this file, leaving only what looks good. And I simply select this file using the FileSelectDialog when starting the robot. Thus it trades the portfolio.


It turns out that Optimization takes 20 minutes. Preview of passes - 3 minutes. Their filter - 3 minutes. Launch of the robot into operation - seconds. The robot does not need to be compiled. Keeping track of versions and charts - similarly.

At startup I can see the Report of each set. At any time I can see the status of the virtual and its namesake from the real, and the hotkey detailed HTML-Report for both the virtual and the real. As for the single set, so for the entire portfolio.


Since the trading statistics of the Virtuals is specially written quickly, it is possible to quickly compare sets on the spot (from the keyboard).

Files:
clip0184.gif  64 kb
1...717273747576777879808182838485...94
New comment
Reason: