How do you know what copy function does? - Expert Advisors and Automated Trading

Alain Verleyen 2013.03.05 11:30 #11

phi.nuts:

Essentially, datetime is just an integer - is a number of second since Jan 1st 1970. When using start time and end time, what Copy function does is to find the closest bar time after start time of given period. Then Copy function will subtract end time with this closest bar time, and divide the result with given period in second. The result is number of bars since start time to end time.

How do you know "what Copy function does" internally ? Is it reliable info from Metaquotes or a guess from you?

Alain Verleyen 2013.03.05 11:59 #12

cowil:
...

My problem however, is when a CopyXXX function is called using both the 'start_time' and 'stop_time' parameters. When time parameters are used, you don't know beforehand how many bars you will be expecting. So whether CopyXXX returns 80 elements (because data still needs to be downloaded/built) or 100 bars (data synced), there's no way (as far as I can work out) you can tell if the retrieved data has been synced or not. So this comes back to the same question again - how do you know that CopyXXX has got the latest data without doing something like CheckLoadHistory() beforehand? Would you have to compare the time of the latest bar in the returned array with say, something like the time returned by TimeTradeServer() and if the two don't approximately match, call CheckLoadHistory()? Or is there some way to establish that CopyXXX has timed out, rather than returned all of the required (i.e. synced) elements? After all, in both instances (when the data is synced or isn't synced) the number of elements will be returned by the CopyXXX function...

The data you didn't (if any), are the oldest. CopyXXX works from right to left (from more recent data up to the old ones).

"When time parameters are used, you don't know beforehand how many bars you will be expecting" : if you copy data to an array it's for using them. Then you have to check your data before using them. Do you have a concrete problem to solve ?

TP and SL ten Array issue... i dont Errors, bugs, questions

phi nuts 2013.03.05 13:41 #13

angevoyageur:

How do you know "what Copy function does" internally ? Is it reliable info from Metaquotes or a guess from you?

Not from MetaQuotes but not guessing either, it's what logical way to do. BTW, I edit my comment, but not the one in your comment, thanks for the critics and correcting it.

angevoyageur:

The data you didn't (if any), are the oldest. CopyXXX works from right to left (from more recent data up to the old ones).

"When time parameters are used, you don't know beforehand how many bars you will be expecting" : if you copy data to an array it's for using them. Then you have to check your data before using them. Do you have a concrete problem to solve ?

1. How do you know "CopyXXX works from right to left (from more recent data up to the old ones)" Is it reliable info from Metaquotes or a guess from you? ;D

2. "When time parameters are used, you don't know beforehand how many bars you will be expecting"", I think you have to re-read cowil comment as whole not some part of it. Cowil did not talking about data in array (the result/return of Copy function), but how Copy function know how many data will be if using start time and end time (the process of Copy function).

My EA program loops Errors, bugs, questions [ARCHIVE!] Any rookie question,

Alain Verleyen 2013.03.05 15:13 #14

phi.nuts:

Not from MetaQuotes but not guessing either, it's what logical way to do. BTW, I edit my comment, but not the one in your comment, thanks for the critics and correcting it.

1. How do you know "CopyXXX works from right to left (from more recent data up to the old ones)" Is it reliable info from Metaquotes or a guess from you? ;D

2. "When time parameters are used, you don't know beforehand how many bars you will be expecting"", I think you have to re-read cowil comment as whole not some part of it. Cowil did not talking about data in array (the result/return of Copy function), but how Copy function know how many data will be if using start time and end time (the process of Copy function).

1. It's in the documentation phi : "It should be noted that elements ordering is from present to past, i.e., starting position of 0 means the current bar." from here. :-D

2. Here, I am a little confused because my poor english, so I trust you.

CopyRates problem Questions from a "dummy" Questions from Beginners MQL5

phi nuts 2013.03.05 17:45 #15

angevoyageur:

1. It's in the documentation phi : "It should be noted that elements ordering is from present to past, i.e., starting position of 0 means the current bar." from here. :-D

2. Here, I am a little confused because my poor english, so I trust you.

Yes, you're right.

Ooh dear, I have to edit my comment again :(

Here's the correct one:

phi.nuts

Essentially, datetime is just an integer - is a number of second since Jan 1st 1970. When using start time and end time, what Copy function logically does is to find the closest bar time after end time of given period. Then Copy function will subtract start time with this closest bar time, and divide the result with given period in second. The result is number of bars since start time to end time.

cowil 2013.03.05 20:37 #16

phi.nuts:

Yes, you're right.

Ooh dear, I have to edit my comment again :(

Here's the correct one:

Wow - quite a conversation while I was asleep! :) Thanks again for your input guys.

phi.nuts - Firstly, I should clarify that I'm wanting to do a CopyXXX of 1M data. So your explanation would work fine if a bar is created for EVERY 1M period of time, regardless of whether a quote comes in over that period of time or not. To refer to a previous post in this discussion, MT4 doesn't create a bar unless at least one quote has come in within that period of time (presume this is the same for MT5?) So basically for your method to work, MetaQuotes would have to guarantee to that a 1M bar is generated for EVERY minute, regardless of whether a quote has come in or not - otherwise the above calculation would be incorrect. This obviously wouldn't be an issue with timeframes larger than 1M, as they're generated from the minute data. However, it's the 1M data that I want to do a CopyXXX on and there are quite often periods or 2-3 minutes when no quotes come in for a symbol...

angevoyageur - Basically what I'm trying to do is find a way to determine whether data returned from a CopyXXX is properly synced and correct (using start/end dates, rather than count) or still needs to be (partially) downloaded. At the moment, I'm doing a CheckLoadHistory() prior to every CopyXXX to ensure that the data is synced etc before a CopyXXX is done.

Counting bars Can a buffer update Are you required to

cowil 2013.03.06 02:56 #17

Hi,

I went back through this post and implemented the recommendations - i.e. comparing CopyXXX's return value with count - if the two were equal, the latest data has been downloaded. However, I was still getting inconsistent data returned so I thought I'd run a little test. I wrote the following script:

string symbol = "CADJPY";
int count = 500;


void OnStart() {

    for (int i=0; i<10; i++) {
        
        datetime times[];
        ArraySetAsSeries(times, true);
        
        int numOfBars = CopyTime(symbol, PERIOD_M1, 0, count, times);
        
        Print("TimeTradeServer: " + TimeTradeServer() + "  times[0]: " + times[0] + "  count: " + count + "  numOfBars: " + numOfBars);
        Sleep(500);
    }    
}

The idea is to set 'symbol' to a currency pair that you don't normally have a chart open for (as the chart obviously downloads the latest data automatically) and then run it. The idea of the 'print' statement is to compare the time of the trading server with the time of the latest available bar. At the same time, the value returned by the CopyXXX command is compared with 'count'. Here the results:

TimeTradeServer: 2013.03.06 03:07:58 times[0]: 2013.03.04 04:44:00 count: 500 numOfBars: 500

TimeTradeServer: 2013.03.06 03:07:58 times[0]: 2013.03.06 03:07:00 count: 500 numOfBars: 500

TimeTradeServer: 2013.03.06 03:07:59 times[0]: 2013.03.06 03:08:00 count: 500 numOfBars: 500

TimeTradeServer: 2013.03.06 03:08:00 times[0]: 2013.03.06 03:08:00 count: 500 numOfBars: 500

You can see that during the the first entry in the log, the data is still getting downloaded. However, 'count' AND 'numOfBars' ARE EQUAL!! So it actually looks like CopyXXX simply returns the number of bars contained in the passed-in array and does nothing as clever as estimate bars once all the data is downloaded etc.

Here's another example - this time with a H1 time period:

NF 0 Test (GBPJPY,H4) 15:51:53 TimeTradeServer: 2013.03.06 04:51:53 times[0]: 2013.02.22 06:00:00 count: 500 numOfBars: 500

QF 0 Test (GBPJPY,H4) 15:51:54 TimeTradeServer: 2013.03.06 04:51:54 times[0]: 2013.03.06 04:00:00 count: 500 numOfBars: 500

So unless I'm completely mistaken, I'm back to my original problem, in that I have to run something like CheckLoadHistory() before doing any form of CopyXXX to ensure that all data is syncronised...

Documentation on MQL5: Timeseries and Indicators Access / Bars

www.mql5.com

Timeseries and Indicators Access / Bars - Documentation on MQL5

Indicator won't fire alerts PREDICT time period iTime and iBarShift return

Alain Verleyen 2013.03.06 07:28 #18

cowil:
...

angevoyageur - Basically what I'm trying to do is find a way to determine whether data returned from a CopyXXX is properly synced and correct (using start/end dates, rather than count) or still needs to be (partially) downloaded. At the moment, I'm doing a CheckLoadHistory() prior to every CopyXXX to ensure that the data is synced etc before a CopyXXX is done.

I looked at the operation of the function CheckLoadHistory(). This could be the solution to your problem :

SeriesInfoInteger(symbol,period,SERIES_SYNCHRONIZED);

phi nuts 2013.03.06 17:43 #19

Cowil, how about try it again, this time delete your historical data completely and if possible use much slower internet connection - I have to use 2G mobile phone which about less than 153 kbps - so it would not download the data at light speed rate.

I use modified version of your code and I got error 4401 (requested history not found) and CopyTime return -1, and I got that error a few second after I attached the script - not immediately error after attached.

  string symbol1 = "GBPJPY";
  int count = 500;
  ENUM_TIMEFRAMES period = PERIOD_H1;
  datetime times[];

  for (int i=0; i<10; i++) 
     {
     PlaySound("wait.wav");
     Print("");
     Print (symbol1);
        
     ArraySetAsSeries(times, true);

     ResetLastError();
     int numOfBars = CopyTime(symbol1, period, 0, count, times);
     Print("Err ",GetLastError());

     Print("TimeTradeServer: " + TimeCurrent() + "  times[0]: " + times[0] + "  count: " + count + "  numOfBars: " + numOfBars);

     Print("bars ",SeriesInfoInteger(symbol1, period, SERIES_BARS_COUNT));
     Print("first bar date ", TimeToString(SeriesInfoInteger(symbol1, period, SERIES_FIRSTDATE), TIME_DATE|TIME_SECONDS));
     Print("last bar date ", TimeToString(SeriesInfoInteger (symbol1, period, SERIES_LASTBAR_DATE),TIME_DATE|TIME_SECONDS));
        
     Sleep(500);
    }    

   for (int i = 0; i < 10; i++)
     {
     Print("",i," time ", TimeToString (times[i],TIME_DATE|TIME_SECONDS));
     }

Number of available bars Getting Started - MetaTrader Instant Execution - Opening

cowil 2013.03.06 22:01 #20

Hi,

Thanks again for your quick responses.

I've had the return codes you mentioned above occasionally, when no data for the Symbol has been downloaded yet and CopyXXX has timed out. However, if you're running an Expert, chances are that the bulk of the data is already downloaded - you only want to update the data. In THAT situation, CopyXXX doesn't give any indication as to whether the data in the array it returns is out of date or up to date. Oh, and I'm afraid I can't do your test with a slow data link as I can't slow my link down at all.

And thanks angevoyageur for your input as well. "SeriesInfoInteger(symbol,period,SERIES_SYNCHRONIZED)" definitely seems to do trick.

Included in the two files are the results of the script phi.nuts provided for two Symbols. The first set of results (first iteration) shows that CopyXXX basically says "all is well", despite having provided out-of-date data in the array it returns. To generate the second set of results, I included the line:

Print("is Synchronised: " + SeriesInfoInteger(symbol1, period, SERIES_SYNCHRONIZED));

in the script. As can been seen, when the data has been properly updated, "SeriesInfoInteger(symbol1, period, SERIES_SYNCHRONIZED)" returns true. The other bizarre thing I noticed is that in the second set of results, the start and end dates of the second and third iteration are already correct but the number of bars (975) doesn't yet reflect the final number of bars (1184).

And in all cases, count == numOfBars...

Files:

Results1.txt 5 kb

Results2.txt 6 kb

Number of available bars Getting characteristics of price Group of OnTester events

Timeseries access and documentation confusion re: CopyXXX() functions... - page 2