CopyRates - BUG - page 3

New comment
 
Stanislav Korotky #:
You can try QuoteRefresh.mqh from the link above and if it returns true (synced timeseries), only then call CopyRates, otherwise skip processing until next request (because otherwise the code will hang anyway). Though, I'm not sure if this solve the issue.

Didnt know this even exists. Thx. But MQL5 only, as it seems?

This backwards loading from right to left is a pain in the a$$. Since with every junk you have to go recalculate all you´ve calculated anyway again. It does not save time, it multiplies it. With a simple MA like MA 1000, you have jumping moving average until all is loaded. 

Don't get me wrong, your tip is good, but all I can do is then: Wait til the last bar is there and then show the panel, show the indication, allow the user to do anything. That might take even a minute, and thats also a problem and forces me to at least show a floating popup to inform the user: "There is no data yet, get a coffee". But yes, better than nothing. Thx.

Next question, how to do in MT4? Or is it compatible?

Edit: I inserted it. 

 
PS: In view of the actual issue itself, I dont think it would solve it. The deadlock occured also in the EA. But when starting only the EA or only the indicator, the data was accessible via CopyRates. But as I said, that I can control using a mutex - even it should not be the case or the job of an EA or indicator. This is basics stuff in multithreading environments to avoid race conditions.
 

In the meanwhile I tried to provoke the race-condition by simple source-code. Not possible. Whatever I do: Deleting history-data-folder (bases/..), running both, EA and indi same time, clicking between timeframes, no result. 

That was my concern. It needs sth else on top to provoke the race, some more complexity, and thats why I did not provide source-code. It just makes no sense.

Nonetheless, here it is:


1. EA

//+------------------------------------------------------------------+
//|                                            CopyRates_Race_EA.mq5 |
//+------------------------------------------------------------------+
//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
  {
  GetTheRates();
   // We have to create extreme conditions
   EventSetMillisecondTimer(1);
   return(INIT_SUCCEEDED);
  }
//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
  {
   //--- destroy timer
   EventKillTimer();
  }
//+------------------------------------------------------------------+
//| Timer function                                                   |
//+------------------------------------------------------------------+
void OnTimer()
  {
  GetTheRates();
  }
  
//+------------------------------------------------------------------+
//| Show output when timespan exceeds n microseconds                 |
//+------------------------------------------------------------------+
void __OutDebug(string funcname, string a1=NULL, uint threshold=1000)
   {
   static ulong t=::GetMicrosecondCount();
   
   ulong t2=::GetMicrosecondCount();
   uint span=(uint)(t2-t);
   if (span>threshold*1000 && a1!=NULL)
      ::Print("(",funcname,") ",a1," ("+(string)(span)+" ms)");   
   t=t2;
   }
//+------------------------------------------------------------------+
//| Copy all rates to provoke race                                   |
//+------------------------------------------------------------------+
void GetTheRates()
   {
   MqlRates rates[];
   __OutDebug(NULL);
   if (CopyRates(Symbol(),PERIOD_CURRENT,0,Bars(Symbol(),PERIOD_CURRENT),rates)<=0)
      __OutDebug(__FUNCTION__,"No data");
   else 
      __OutDebug(__FUNCTION__, "Threshold exceeded", 10000);
   }

2. Indicator


//+------------------------------------------------------------------+
//|                                     CopyRates_Race_Indicator.mq5 |
//+------------------------------------------------------------------+
#property indicator_chart_window
//+------------------------------------------------------------------+
//| Custom indicator initialization function                         |
//+------------------------------------------------------------------+
int OnInit()
  {
  GetTheRates();
  // Create extreme condition
  EventSetMillisecondTimer(1);
  return(INIT_SUCCEEDED);
  }
//+------------------------------------------------------------------+
//| Custom indicator iteration function                              |
//+------------------------------------------------------------------+
int OnCalculate(const int32_t rates_total,
                const int32_t prev_calculated,
                const datetime &time[],
                const double &open[],
                const double &high[],
                const double &low[],
                const double &close[],
                const long &tick_volume[],
                const long &volume[],
                const int32_t &spread[])
  {
//---
   
//--- return value of prev_calculated for next call
   return(rates_total);
  }
//+------------------------------------------------------------------+
//| Timer function                                                   |
//+------------------------------------------------------------------+
void OnTimer()
  {
  GetTheRates(); 
  }
//+------------------------------------------------------------------+
//+------------------------------------------------------------------+
//| Show output when timespan exceeds n microseconds                 |
//+------------------------------------------------------------------+
void __OutDebug(string funcname, string a1=NULL, uint threshold=1000)
   {
   static ulong t=::GetMicrosecondCount();
   
   ulong t2=::GetMicrosecondCount();
   uint span=(uint)(t2-t);
   if (span>threshold*1000 && a1!=NULL)
      ::Print("(",funcname,") ",a1," ("+(string)(span)+" ms)");   
   t=t2;
   }
//+------------------------------------------------------------------+
//| Copy all rates to provoke race                                   |
//+------------------------------------------------------------------+
void GetTheRates()
   {
   MqlRates rates[];
   __OutDebug(NULL);
   if (CopyRates(Symbol(),PERIOD_CURRENT,0,Bars(Symbol(),PERIOD_CURRENT),rates)<=0)
      __OutDebug(__FUNCTION__,"No data");
   else 
      __OutDebug(__FUNCTION__, "Threshold exceeded", 10000);
   }
 
Alain Verleyen #:

So the be clearer, the documentation you pointed says :

Did you checked that it can actually lead to deadlock ?

If yes, can you define "the possibility is high" ? And also in which (indicator) context it can happen ?

I did not test for deadlocks as it's inappropriate way of timeseries access as per MQ, I'd rather test what should work normally. The category "high" comes from MQ, so it's better to ask them for the answer.
 
Doerk Hilger #:

This backwards loading from right to left is a pain in the a$$. Since with every junk you have to go recalculate all you´ve calculated anyway again. It does not save time, it multiplies it. With a simple MA like MA 1000, you have jumping moving average until all is loaded. 

Don't get me wrong, your tip is good, but all I can do is then: Wait til the last bar is there and then show the panel, show the indication, allow the user to do anything. That might take even a minute, and thats also a problem and forces me to at least show a floating popup to inform the user: "There is no data yet, get a coffee". But yes, better than nothing. Thx.

The question is how many bars do you request. If MT5 settings are for unlimited number of bars on charts and you ask CopyRates(...,Bars()), in many MQL-programs in parallel, then it can require some memory allocations inbetween. Probably you need an input to limit the number of bars to process.

Why do you need the same timeseries in indicator and EA? Normally you design a group of programs using the principle "divide resposibility", so only one of them uses raw big data to get intermediate signals, and then pass these relatively small signals to EA.

 
Stanislav Korotky #:
I did not test for deadlocks as it's inappropriate way of timeseries access as per MQ, I'd rather test what should work normally. The category "high" comes from MQ, so it's better to ask them for the answer.

Thanks.

From my experience there is no such deadlock, that's why I asked you.

Stanislav Korotky #:

The question is how many bars do you request. If MT5 settings are for unlimited number of bars on charts and you ask CopyRates(...,Bars()), in many MQL-programs in parallel, then it can require some memory allocations inbetween. Probably you need an input to limit the number of bars to process.

In the first post, Dirk said shift=345, count=1.

 
Alain Verleyen #:

Thanks.

From my experience there is no such deadlock, that's why I asked you.

In the first post, Dirk said shift=345, count=1.

The question was primarily not about function parameters, but about terminal settings, because even if you request a single bar yet terminal is instructed to download unlimited history - you'll be waiting for the entire history download before you'll finally get your single bar.

As for deadlocks, I suppose it's more probable under stress conditions, for example when a lot of MQL-programs request a lot of data during high frequency market updates. It's hard to reproduce, even Dirk did not provide a test case.

123
New comment