iClose(), iTime(), CopyRates() etc. takes seconds and crashes EA and MT5 - page 2

New comment
 

Hey Alain,

of course, thank you very much. I stepped a bit deeper into it and will try to help you to understand the procedure. The problem mainly occurs after the timeframe just has changed, but not only. And we talk about a very general problem - behind - CopyRates(), CopyTime(), iTime() etc. which seem to occur in specific environments, cause approximately only 1 of 100 users report this, I am not sure if you will be able to reproduce the behaviour.

Procedure: 

When the timeframe is changed, the main EA broadcasts a message to all its modules in a message queue which is executed during OnTimer(), and these modules use iTime() etc. to re-initialize. What I suspect is, that there is a general issue when functions like CopyRates() etc. request plenty of quote data during OnTimer(). 

Below you will find a code which was able to reproduce the problem on a future live account of AMP, using EURUSD (with a shift value of zero - no history data requested!) when switching quickly between timeframes this morning. I am using Windows Server 2012 with 8 GB of RAM on a VPS system. 

 

Remarks:

- It never happens with MT4 - code is 1:1

- I am not sure if it happens with demo accounts

Code:

//--- Var tested in OnTimer()
bool timeframechanged=false;

//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
  {
      //--- Detect timeframe change, will be used in OnTimer()
      static ENUM_TIMEFRAMES tfold=WRONG_VALUE;
      ENUM_TIMEFRAMES tf=::Period();
      
      if (tf!=tfold)
         timeframechanged=true;

   EventSetMillisecondTimer(250);
   return(INIT_SUCCEEDED);
  }
//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
  {
   EventKillTimer();
  }
//+------------------------------------------------------------------+
//| Timer function                                                   |
//+------------------------------------------------------------------+
void OnTimer()
  {
      //--- Request some quotes when timeframe has changed
      if (timeframechanged)
         {
         string symbol=::Symbol();
         for (int i=0;i<100;i++)
            {
            datetime dt=_iTime(symbol,PERIOD_M1);
            dt=_iTime(symbol,PERIOD_H1);
            dt=_iTime(symbol,PERIOD_CURRENT);
            MqlRates r=_iRates(symbol,PERIOD_M15);
            }
         timeframechanged=false;
         }
  }


//--- When a request took longer than 1 second, there is a potential 
//    problem with server/connection. To avoid a crash, any copying 
//    of quote data is suspended for some seconds

long quotes_suspend_time=0;
#define QUOTES_MEASURE_TIME_INIT(retval) long t_begin = ::GetTickCount(); if (quotes_suspend_time>0 && t_begin-quotes_suspend_time<4000) return retval; 
#define QUOTES_MEASURE_TIME_UPDATE(shift) long t_measure = ::GetTickCount(); quotes_suspend_time=0; if (t_measure-t_begin>1000) { quotes_suspend_time = t_measure; Print("Critical error: Timeout reading bar data! (",__FUNCTION__,"/",shift,")"); }


datetime   _iTime(string symbolname, ENUM_TIMEFRAMES tf, int shift=0)
   {
      //--- Get out with error when suspended
      QUOTES_MEASURE_TIME_INIT(0)
      datetime result=iTime(symbolname,tf,shift);
      QUOTES_MEASURE_TIME_UPDATE(shift)
      return result;
   }
   
   
MqlRates _iRates(string symbolname, ENUM_TIMEFRAMES tf, int shift=0)
   {
      MqlRates rates[];

      if (!_CopyRates(symbolname,tf,shift,1,rates))
	{
	MqlRates r={0,0,0,0,0,0,0,0};
        return r;
        }
      return rates[0];
           
   }
   
   
bool _CopyRates(string symbolname, ENUM_TIMEFRAMES tf, int shift, int cnt, MqlRates &rates[])
   {
      //--- Get out with error when suspended
      QUOTES_MEASURE_TIME_INIT(false)
      //---
      bool result=CopyRates(symbolname,tf,shift,cnt,rates)==cnt;
      QUOTES_MEASURE_TIME_UPDATE(shift+cnt)
      return result;
   }
 
Doerk Hilger:

Hey Alain,

of course, thank you very much. I stepped a bit deeper into it and will try to help you to understand the procedure. The problem mainly occurs after the timeframe just has changed, but not only. And we talk about a very general problem - behind - CopyRates(), CopyTime(), iTime() etc. which seem to occur in specific environments, cause approximately only 1 of 100 users report this, I am not sure if you will be able to reproduce the behaviour.

Procedure: 

When the timeframe is changed, the main EA broadcasts a message to all its modules in a message queue which is executed during OnTimer(), and these modules use iTime() etc. to re-initialize. What I suspect is, that there is a general issue when functions like CopyRates() etc. request plenty of quote data during OnTimer(). 

Below you will find a code which was able to reproduce the problem on a future live account of AMP, using EURUSD (with a shift value of zero - no history data requested!) when switching quickly between timeframes this morning. I am using Windows Server 2012 with 8 GB of RAM on a VPS system. 

 

Remarks:

- It never happens with MT4 - code is 1:1

- I am not sure if it happens with demo accounts

Code:

I can test on a live account of AMP.

"This morning", we are Saturday, so it happens only when there is no ticks ? or doesn't matter ?

Forum on trading, automated trading systems and testing trading strategies

iClose(), iTime(), CopyRates() etc. takes seconds and crashes EA and MT5

Doerk Hilger, 2019.05.29 12:52

Thanks, I updated the function.

But anyway, this must be a kind of a bug, because I still have this issue and it happens from time to time and it happens when the Shift-value is ZERO.

This morning, EA is running since 48 hours, just like that in the middle of nothing, everything freezes and the function reports a timeout at _iTime(symbol,PERIOD_CURRENT,0). This should never happen, in no circumstance. 

The timeframe change you talked about now is to force the update, trying to get the problem, or is it also in real environment ? You never talked about changing timeframe up to now, so I would like to be sure.
 
 The change of the timeframe seems to be a method to force the issue, yes, but it also happens without such a change and during market hours as well as on weekendes. I just recognized this recently, thats why I mention the change of timeframes additionaly. 
 
Doerk Hilger:
 The change of the timeframe seems to be a method to force the issue, yes, but it also happens without such a change and during market hours as well as on weekendes. I just recognized this recently, thats why I mention the change of timeframes additionaly. 
I tested changing timeframe, I was not able to reproduce the problem. I will see when the market will be open.
 
I could also modify the test code in a way that you can let it run for hours and wait til it happens. But it will, sooner or later and the matter is, that this issue is a serious problem which exists at least since a year.
 

New testcode for infinite loop


//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
  {
   EventSetMillisecondTimer(1000);
   return(INIT_SUCCEEDED);
  }
//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
  {
   EventKillTimer();
  }
//+------------------------------------------------------------------+
//| Timer function                                                   |
//+------------------------------------------------------------------+
void OnTimer()
  {
      //--- Request some quotes when timefram has changed
      string symbol=::Symbol();
      for (int i=0;i<100;i++)
         {
         double c=_iClose(symbol,PERIOD_M1,i);
         c=_iClose(symbol,PERIOD_H1,i);
         c=_iClose(symbol,PERIOD_CURRENT,i);
         }
  }


double   _iClose(string symbolname, ENUM_TIMEFRAMES tf, int shift=0)
   {
      return _iRates(symbolname,tf,shift).close;
   }
   
MqlRates _iRates(string symbolname, ENUM_TIMEFRAMES tf, int shift=0)
   {
      //--- Use only static vars to save time
      static MqlRates rates[1];

      if (!_CopyRates(symbolname,tf,shift,1,rates))
         {
         MqlRates r={0,0,0,0,0,0,0,0};
         return r;
         }
      return rates[0];
   }
   
   
bool _CopyRates(string symbolname, ENUM_TIMEFRAMES tf, int shift, int cnt, MqlRates &rates[])
   {
      static long tc=-1;
      static bool quotes_request=true;
      
      bool result=false;
      while (quotes_request && !result)
         {
         tc=::GetTickCount();
         result=CopyRates(symbolname,tf,shift,cnt,rates)==cnt;
         if (!result)
            {
            //--- Took longer than 1 second?
            if (GetTickCount()-tc>1000)
               {
               //--- Allow retry at timeout
               if (!__CriticalErrorQuotes(__FUNCTION__,symbolname,tf,shift))
                  {
                  //--- Suspend and terminate
                  quotes_request=false;
                  ::ExpertRemove();
                  return false;
                  }
               }
            else
               //--- Just no data
               return false;
            }
         }
      return result;
   }      
   
bool __CriticalErrorQuotes(string funcname, string symbol, ENUM_TIMEFRAMES tf, int shift)
   {
      if (symbol==NULL)
         symbol=::Symbol();
      string msg=funcname+"/"+symbol+"/"+(string)tf+"/"+(string)shift;
      return (::MessageBox("Timeout reading quote data!\n("+msg+")\n\nPress 'Retry' to resume or 'Cancel' to terminate execution.","Critical error!",MB_RETRYCANCEL|MB_ICONSTOP)!=IDCANCEL);
   }
  
 

By the way, I´d be very happy if some more people could use this code and let it run for some hours, because this is an issue which probably affects ANY expert and indicator - sooner or later.

No matter if you ever recognized this problem or not, I have a huge number of users and a few of them have this issue. So it´s just a matter of time, when your EA or indicator is affected too.

Thank you!

 

It may also be related to memory.

Have you looked into the EVENTVWR system log ?

It could show entries like Terminal.exe - 'APPLICATION_HANG' and give you an error code which you can Google to find out what exactly went wrong.

 
Doerk Hilger:

New testcode for infinite loop


Launched.
 
Marco vd Heijden:

It may also be related to memory.

Have you looked into the EVENTVWR system log ?

It could show entries like Terminal.exe - 'APPLICATION_HANG' and give you an error code which you can Google to find out what exactly went wrong.

Thank you, never did but will have a look at it next time.

Alain Verleyen:
Launched.


Thanx :)


123
New comment