How to check for a specific data before download

[Gustavo Enedir Hennemann]

Hello,

In some cases, when a try to copy a price, open price for example, the time to return is too long.

This is a part of the code. Here I want to copy just one open price (one day). It works well, but sometimes it takes too much time to return an error. Even when the data doesn't exist.

nOpen = CopyOpen(symbol, PERIOD_D1, symbolDate, symbolDate, open_array);

So, I'm trying to check if the date exists before to try to get it. Something like this, but more specific:

firstServerDate = SeriesInfoInteger(_Symbol, PERIOD_D1, SERIES_SERVER_FIRSTDATE);

The problem is the error returned is the same for comunications problems, time out and no data available. Always nOpen = 1 and GetLastError() = 4401.

Are there a function that return any value indicating if a specific date exists? Or an other alternative and fast way?

Thanks in advance.

[grandaevus]

Gustavo Hennemann:

Hello,

In some cases, when a try to copy a price, open price for example, the time to return is too long.

This is a part of the code. Here I want to copy just one open price (one day). It works well, but sometimes it takes too much time to return an error. Even when the data doesn't exist.

So, I'm trying to check if the date exists before to try to get it. Something like this, but more specific:

The problem is the error returned is the same for comunications problems, time out and no data available. Always nOpen = 1 and GetLastError() = 4401.

Are there a function that return any value indicating if a specific date exists? Or an other alternative and fast way?

Thanks in advance.

1) 

nOpen = CopyOpen(symbol, PERIOD_D1, symbolDate, 1, open_array);

would be better if you want to get just one single data.

2) use static array not dynamic.

datetime open_array[1];

3) Is there any line in your code that uses CopyBuffer function to get data from an indicator?

if yes, be sure that handle of that indicator has been created inside OnInit() 

[Gustavo Enedir Hennemann]

grandaevus:
1)

nOpen = CopyOpen(symbol, PERIOD_D1, symbolDate, 1, open_array);

would be better if you want to get just one single data.

2) use static array not dynamic.

datetime open_array[1];

3) Is there any line in your code that uses CopyBuffer function to get data from an indicator?

if yes, be sure that handle of that indicator has been created inside OnInit() 

Hi @grandaevus, thanks for your comments,

My considerations:

1) This way works better, much less errors and faster. But same final result. I couldn't test communication fails to check the returned codes. Are there different codes for error and "data not available" that works?

2) Makes no difference. Was any difference expected?

3) No, I'm not using CopyBuffer().

PS: This code is inside a script not EA. So, all variables are global.

Thanks again

[Alain Verleyen]

Gustavo Hennemann:

Hi @grandaevus, thanks for your comments,

My considerations:

1) This way works better, much less errors and faster. But same final result. I couldn't test communication fails to check the returned codes. There are different codes for error and "data not available" that works?

2) Makes no difference. Was any difference expected?

3) No, I'm not using CopyBuffer().

PS: This code is inside a script not EA. So, all variables are global.

Thanks again

Your problem is unclear. What is "too much time..." ?

What are you trying to do ?

[Gustavo Enedir Hennemann]

Alain Verleyen:

Your problem is unclear. What is "too much time..." ?

What are you trying to do ?

Hi @Alain Verleyen,

The point is: I'm trying to get the opened price for a list of assets, some of them have price in the date I want some of them no. To get the price MetaTrader needs some time (around 10 or 20 seconds when there is no local data), and when the asset have no negociation in that date (there is no data at all) MetaTrader also takes 10 or 20 seconds in each asset.
I will wonder if there is a way to figure out if that date for the asset have negociation, in other words, I'm trying to do the task as fast as possible (the list is big).

[Stanislav Korotky]

Maybe you should check against SERIES_FIRSTDATE (not SERIES_SERVER_FIRSTDATE), because you're interested in fast access to data available on the client, not on the server.

[Gustavo Enedir Hennemann]

Stanislav Korotky:
Maybe you should check against SERIES_FIRSTDATE (not SERIES_SERVER_FIRSTDATE), because you're interested in fast access to data available on the client, not on the server.

Hello,

This function doesn't tell me if there are data in a specific date, also it takes the same 10 or 20 seconds if I check the server. If I check local and after that check the server, it will be 2 checks and more time.

[Stanislav Korotky]

Gustavo Hennemann:

Hello,

This function doesn't tell me if there are data in a specific date, also it takes the same 10 or 20 seconds if I check the server. If I check local and after that check the server, it will be 2 checks and more time.

SERIES_FIRSTDATE should not check the server. So there should not be the same time overheads as mentioned above.

Knowing the first date you do know if a specific date is covered or not.

You asked about a local data checkup ASAP. This method provides you with a way to output existing data promptly, and optionally request postponded refresh if you need deeper history from server.

If it's not what you're asking for, please, clarify your requirements.

[Alain Verleyen]

Gustavo Hennemann:

Hi @Alain Verleyen,

The point is: I'm trying to get the opened price for a list of assets, some of them have price in the date I want some of them no. To get the price MetaTrader needs some time (around 10 or 20 seconds when there is no local data), and when the asset have no negociation in that date (there is no data at all) MetaTrader also takes 10 or 20 seconds in each asset.
I will wonder if there is a way to figure out if that date for the asset have negociation, in other words, I'm trying to do the task as fast as possible (the list is big).

CopyOpen() takes 10 or 20 seconds to return ?

Which build are you using ?

Did you try Stanislav proposal ?

[Gustavo Enedir Hennemann]

This is the actual code:

   double   open_array[];
   int      qtData;
   string   strValue;
   datetime firstServerDate = 0;
   
   //all symbols in market watch
   symbolsTotal = SymbolsTotal(true);
   for(i=0; i<symbolsTotal && !IsStopped(); i++){
      bool  downOk = false;
      bool  dateOk = false;
      int   error;
      symbol = SymbolName(i, true);
      qtData = 0;
      
      //check first date in server
      for(j=0; (j<inpTimeOutDownload) && !dateOk && !IsStopped(); j++){
         dateOk = SeriesInfoInteger(symbol, PERIOD_D1, SERIES_SERVER_FIRSTDATE, firstServerDate);
         if(!dateOk){
            Sleep(1000);
         }
      }
      
      //data exists?
      error = 0;
      if(firstServerDate <= inpSymbolDate){   //date I want to get
         
         //try to get the data
         for(j=0; (j<inpTimeOutDownload && qtData<=0) && !IsStopped(); j++){
            ResetLastError();
            qtData = CopyOpen(symbol, PERIOD_D1, inpSymbolDate, 1, open_array);
            error = GetLastError();
            if(qtData<=0 || error==4401){
               Sleep(1000);
            }
         }
         
         //error
         if(j==inpTimeOutDownload || qtData<0){
            strValue = "0,000";
            printf("error in CopyOpen %s", symbol);
         }
         else{
            strValue = StringFormat("%.3f", open_array[0]);
            StringReplace(strValue, ".", ",");
         }
      }
      else{
         strValue = "0,000";
         printf("asset don't have data in this date %s", symbol);
      }
      list.addElement(SymbolName(i, true) + "\t" + strValue);
      Sleep(5);         //wait to not freeze the system
   }

[Marcel Fitzner]

Since the OP has still not been answered - and I finally saw one of those crazy cases where my broker did not properly send the history of a symbol, I'd like to revive this thread.

One pic shall explain what I mean:

With a request (like Gustavo did):

dateOk = SeriesInfoInteger(symbol, PERIOD_D1, SERIES_SERVER_FIRSTDATE, firstServerDate);

firstServerDate is exactly the first date as shown in the chart (06.01.2012) and Bars("CRDI.IT", PERIOD_D1) will return ~2300 bars while the history - where actual values are available through CopyTime, CopyRates and so forth - starts at the 22.09.2016. (Btw. between that point and now are around 1100 bars)

For some reason CopyTime("CRDI.IT", PERIOD_D1, 2300, 1, times) (with times being a static array with just 1 element) hangs there for ~4 minutes(!!) until it eventually figures out that there is no data available and an error 4401 is produced.

To me the main question remains:

   Is there a smarter way to fetching the actually available history and avoid waiting for a lengthy time or is this the best possible way already? 

Maybe this is/was the case for other trader coder friends here as well. If you happen to have any good advice, I'd appreciate your help much!

This is the method I've been working with and which checks the server history:

bool CheckServerHistory(const string symbol, const ENUM_TIMEFRAMES period, const int barsToTryToDownload, bool ignorePartialHistory = true)
{
        datetime first_server_date = 0;
        uint startTickCount = GetTickCount();

        Print("Fetching ServerFirstDate...");
        while (!SeriesInfoInteger(symbol, PERIOD_M1, SERIES_SERVER_FIRSTDATE, first_server_date) && !IsStoppedExt(startTickCount))
                Sleep(5);
        
        Print("First server date: ", first_server_date);

        //Enough data on server?
        if (!ignorePartialHistory && first_server_date>TimeCurrent()-barsToTryToDownload*PeriodSeconds(period))
                return false;
        
        //load data step by step
        int fail_cnt = 0;
        datetime times[1];

        uint startTickCount_outer = GetTickCount();
        uint startTickCount_inner;
        
        Print("Checking SERIES_SYNCHRONIZED...");
        while (!IsStoppedExt(startTickCount_outer) && fail_cnt++ < 10)
        {
                startTickCount_inner = GetTickCount();
                
                while (!SeriesInfoInteger(symbol,period,SERIES_SYNCHRONIZED) && !IsStoppedExt(startTickCount_inner))
                        Sleep(5);
                
                Print("ask for built bars...");
                int bars = Bars(symbol, period);
                if (bars > barsToTryToDownload)
                        return true;
                
                PrintFormat("Copy Time... barsToTryToDownload-1: %d, Symbol:%s", barsToTryToDownload-1, symbol);
                int copied = CopyTime(symbol, period, barsToTryToDownload-1, 1, times);
                
                if (copied == 1)
                        return true;
                else {
                        if (copied == -1)
                                PrintFormat("Could not copy 1 bar at bar #%d! Error : %d", barsToTryToDownload-1, GetLastError());
                        Sleep(10);
                }
        }
        
        return false;
}

Cheers,

Marcel