Timeseries access and documentation confusion re: CopyXXX() functions...
Hi cowil,
I am not programmer so I do not know the answer. I hope that some members with programming knowledge will reply sorry.
Hi,
I'm experiencing a bit of "documentation confusion" I hope someone can help me with. The documentation for all CopyXXX() functions contains the following note:
"Note
When requesting data from the indicator, if requested timeseries are not yet built or they need to be downloaded from the server, the function will immediately return -1, but the process of downloading/building will be initiated.
When requesting data from an Expert Advisor or script, downloading from the server will be initiated, if the terminal does not have these data locally, or building of a required timeseries will start, if data can be built from the local history but they are not ready yet. The function will return the amount of data that will be ready by the moment of timeout expiration. "
The confusion I'm experiencing concerns the last sentence - i.e. what is the timeout expiration related to (i.e. is this the timeout of the CopyXXX() function or a timeout for downloading data or something different again?) and how does CopyXXX() know the amount of data that will be ready if it still requires downloading?
Also, I'm a bit confused as to why MetaQuotes chose to simply return a -1 if a CopyXXX() function is called within an Indicator and data requires downloading/assembling, while the equivalent of a CheckLoadHistory() function needs to be written (https://www.mql5.com/en/docs/series/timeseries_access#synchronized) and called every time a CopyXXX() function is called within an Expert or Script? Why not simply return a -1 within Experts and Scripts as well? Or have I got this completely wrong?
Any help on this would be greatly appreciated.
1. In this case, time out expiration is related to Copy function, not downloading or something else.
2. Copy function knows how much data needed to be downloaded because MT5 knows the current server time and compared with last known of last bar time.
3. Yes, Copy function can simply return -1 in EA/Script. CheckLoadHistory() in mql5 documentation is a user defined function (that is created by programmer) and not an mql5 function.
4. Don't forget to check GetLastError().
1. In this case, time out expiration is related to Copy function, not downloading or something else.
2. Copy function knows how much data needed to be downloaded because MT5 knows the current server time and compared with last known of last bar time.
3. Yes, Copy function can simply return -1 in EA/Script. CheckLoadHistory() in mql5 documentation is a user defined function (that is created by programmer) and not an mql5 function.
4. Don't forget to check GetLastError().
Hi,
Firstly, thanks for the response.
Re: point 2, I can see how comparing the dates would provide the total amount of data accurately for larger time frames. However, on smaller time frames (especially 1m), there may be no quote over a 1m period. I know that in MT4, no bar would be created for that particular 1m period. Does MT5 guarantee that a bar will be created for every minute period, even if no quote is received over that time? If this is not the case, than the number of bars returned by CopyXXX would be incorrect if that data is not yet available locally?
Re: point 3. According to the documentation, a -1 is returned (if CopyXXX is called within an Expert) when the requested data falls either outside the database of the Trading Server or outside the MAX_BARS setting of the Terminal. If the data needs to be downloaded or "assembled" locally, the correct number of bars is returned by the function, even though THAT DATA MAY NOT YET BE AVAILABLE LOCALLY. My understanding of the documentation is that a user defined function such as CheckLoadHistory() is required to ensure that the required data has been downloaded and assembled, prior to the CopyXXX being called - otherwise CopyXXX may contain only some of the requested data. Please correct me if I'm wrong on this as I'd LOVE to not have to call CheckLoadHistory() before every CopyXXX() (makes optimising very sllloooowwwww).
Thanks again for your help on this.
- www.mql5.com
cowil:...
what is the timeout expiration related to (i.e. is this the timeout of the CopyXXX() function or a timeout for downloading data or something different again?) and how does CopyXXX() know the amount of data that will be ready if it still requires downloading?
My experience with reading the docs is that their English is not the best, so when they say something that is a little confusing, like "the data that will be ready", you have to step back and ask how would they have said that if they spoke English natively? My interpretation, if I'm right in this case, is that they mean "the data that is ready". There is so much of this kind of thing in their documentation, they really should hire native English speakers to proof their docs (people who understand trading and coding, not just some translator who needs a few bucks.)
Hi,
Firstly, thanks for the response.
Re: point 2, I can see how comparing the dates would provide the total amount of data accurately for larger time frames. However, on smaller time frames (especially 1m), there may be no quote over a 1m period. I know that in MT4, no bar would be created for that particular 1m period. Does MT5 guarantee that a bar will be created for every minute period, even if no quote is received over that time? If this is not the case, than the number of bars returned by CopyXXX would be incorrect if that data is not yet available locally?
Re: point 3. According to the documentation, a -1 is returned (if CopyXXX is called within an Expert) when the requested data falls either outside the database of the Trading Server or outside the MAX_BARS setting of the Terminal. If the data needs to be downloaded or "assembled" locally, the correct number of bars is returned by the function, even though THAT DATA MAY NOT YET BE AVAILABLE LOCALLY. My understanding of the documentation is that a user defined function such as CheckLoadHistory() is required to ensure that the required data has been downloaded and assembled, prior to the CopyXXX being called - otherwise CopyXXX may contain only some of the requested data. Please correct me if I'm wrong on this as I'd LOVE to not have to call CheckLoadHistory() before every CopyXXX() (makes optimising very sllloooowwwww).
Thanks again for your help on this.
1. If "no quote receive" is caused by no connection to broker server, then once there's connection, MT5 will update the M1 data and - if needed - built data for higher TF. However, my experience show that this will takes time, about a minute or less, and so during that time Copy function will return -1.
2. I highlighted some of your comment which is incorrect. The correct one according to doc is
The function will return the amount of data that will be ready by the moment of timeout expiration, but history downloading will continue, and at the next similar request the function will return more data.
When Copy function return is -1, then there must be something wrong with the historical data, the Copy function will return the amount of data that will be ready at timeout, that does not mean Copy function will return the correct number of copied data - it only return the data that will be ready at timeout.
So it will be good idea, when Copy function return -1, check with GetLastError() and SeriesInfoInteger(), then you can do something about it, like using CheckLoadHistory or else.
During testing or optimization, there's no need for checking the data, because tester already try to use a complete data and it will try to download new data if necessary. So on that case, you may have to add some code in OnTester() to disable CheckLoadHistory().
Hope that's helps.
1. If "no quote receive" is caused by no connection to broker server, then once there's connection, MT5 will update the M1 data and - if needed - built data for higher TF. However, my experience show that this will takes time, about a minute or less, and so during that time Copy function will return -1.
2. I highlighted some of your comment which is incorrect. The correct one according to doc is
When Copy function return is -1, then there must be something wrong with the historical data, the Copy function will return the amount of data that will be ready at timeout, that does not mean Copy function will return the correct number of copied data - it only return the data that will be ready at timeout.
So it will be good idea, when Copy function return -1, check with GetLastError() and SeriesInfoInteger(), then you can do something about it, like using CheckLoadHistory or else.
During testing or optimization, there's no need for checking the data, because tester already try to use a complete data and it will try to download new data if necessary. So on that case, you may have to add some code in OnTester() to disable CheckLoadHistory().
Hope that's helps.
Hi,
Thanks again for the info - much appreciated.
Re: your following comment:
" When Copy function return is -1, then there must be something wrong with the historical data, the Copy function will return the amount of data that will be ready at timeout, that does not mean Copy function will return the correct number of copied data - it only return the data that will be ready at timeout."
I'm still a little confused. As I'm sure you're aware, CopyXXX normally returns an int, containing either the number of copied elements or a -1 if an error occurred. My understanding of the above statement is that CopyXXX will return both a -1 AND the amount of data that will be ready at timeout if the historical data requires downloading and/or building. Obviously CopyXXX can't return both a -1 and the amount of data (i.e. number of copied elements) at the same time as the function only returns one int - not two. So I'm presuming that if CopyXXX returns a -1, the size of the target array can be used to work out the number of bars that will be ready at timeout? Or have I got this wrong?
Hi,
Thanks again for the info - much appreciated.
Re: your following comment:
" When Copy function return is -1, then there must be something wrong with the historical data, the Copy function will return the amount of data that will be ready at timeout, that does not mean Copy function will return the correct number of copied data - it only return the data that will be ready at timeout."
I'm still a little confused. As I'm sure you're aware, CopyXXX normally returns an int, containing either the number of copied elements or a -1 if an error occurred. My understanding of the above statement is that CopyXXX will return both a -1 AND the amount of data that will be ready at timeout if the historical data requires downloading and/or building. Obviously CopyXXX can't return both a -1 and the amount of data (i.e. number of copied elements) at the same time as the function only returns one int - not two. So I'm presuming that if CopyXXX returns a -1, the size of the target array can be used to work out the number of bars that will be ready at timeout? Or have I got this wrong?
Sorry, cowil, I got it wrong :(
It should be either -1 or amount of data that will be ready at timeout. When Copy function return amount of data that will be ready at timeout, we should not work with whatever in array. When timeout occurs, call Copy function again then we can work with the data in array, though I prefer not to, if the amount of data is less than the one I requested.
Quite funny though, that Copy function return 2 value at the same time :).
Sorry, cowil, I got it wrong :(
It should be either -1 or amount of data that will be ready at timeout. When Copy function return amount of data that will be ready at timeout, we should not work with whatever in array. When timeout occurs, call Copy function again then we can work with the data in array, though I prefer not to, if the amount of data is less than the one I requested.
Quite funny though, that Copy function return 2 value at the same time :).
No probs re the error. And yeah, I was also thinking that CopyXXX would be quite a tricky function if it could return 2 ints at once. :)
Soooo... if I understand it correctly, when a CopyXXX function is called and the data hasn't been synced, either a -1 or the amount of data that will be ready at timeout is returned. However, the retrieved data obviously shouldn't be used as it will be incomplete (i.e. the latest data will still need to be downloaded/built).
The above all works well enough if a specific number of bars are requested (using the count parameter). For instance, if you specify a count of 100 and CopyXXX returns 80 (80 bars available at timeout), you know that an update is required.
My problem however, is when a CopyXXX function is called using both the 'start_time' and 'stop_time' parameters. When time parameters are used, you don't know beforehand how many bars you will be expecting. So whether CopyXXX returns 80 elements (because data still needs to be downloaded/built) or 100 bars (data synced), there's no way (as far as I can work out) you can tell if the retrieved data has been synced or not. So this comes back to the same question again - how do you know that CopyXXX has got the latest data without doing something like CheckLoadHistory() beforehand? Would you have to compare the time of the latest bar in the returned array with say, something like the time returned by TimeTradeServer() and if the two don't approximately match, call CheckLoadHistory()? Or is there some way to establish that CopyXXX has timed out, rather than returned all of the required (i.e. synced) elements? After all, in both instances (when the data is synced or isn't synced) the number of elements will be returned by the CopyXXX function...
- www.mql5.com
No probs re the error. And yeah, I was also thinking that CopyXXX would be quite a tricky function if it could return 2 ints at once. :)
Soooo... if I understand it correctly, when a CopyXXX function is called and the data hasn't been synced, either a -1 or the amount of data that will be ready at timeout is returned. However, the retrieved data obviously shouldn't be used as it will be incomplete (i.e. the latest data will still need to be downloaded/built).
The above all works well enough if a specific number of bars are requested (using the count parameter). For instance, if you specify a count of 100 and CopyXXX returns 80 (80 bars available at timeout), you know that an update is required.
My problem however, is when a CopyXXX function is called using both the 'start_time' and 'stop_time' parameters. When time parameters are used, you don't know beforehand how many bars you will be expecting. So whether CopyXXX returns 80 elements (because data still needs to be downloaded/built) or 100 bars (data synced), there's no way (as far as I can work out) you can tell if the retrieved data has been synced or not. So this comes back to the same question again - how do you know that CopyXXX has got the latest data without doing something like CheckLoadHistory() beforehand? Would you have to compare the time of the latest bar in the returned array with say, something like the time returned by TimeTradeServer() and if the two don't approximately match, call CheckLoadHistory()? Or is there some way to establish that CopyXXX has timed out, rather than returned all of the required (i.e. synced) elements? After all, in both instances (when the data is synced or isn't synced) the number of elements will be returned by the CopyXXX function...
Essentially, datetime is just an integer - is a number of second since Jan 1st 1970. When using start time and end time, what Copy function logically does is to find the closest bar time after start end time of given period. Then Copy function will subtract end start time with this closest bar time, and divide the result with given period in second. The result is number of bars since start time to end time.
<edit : added the words: "logically", strike out the word "start" and "end" and add the word "end " and "start", see explanation on the next following comments. Many Thanks to angevoyageur>
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Hi,
I'm experiencing a bit of "documentation confusion" I hope someone can help me with. The documentation for all CopyXXX() functions contains the following note:
"Note
When requesting data from the indicator, if requested timeseries are not yet built or they need to be downloaded from the server, the function will immediately return -1, but the process of downloading/building will be initiated.
When requesting data from an Expert Advisor or script, downloading from the server will be initiated, if the terminal does not have these data locally, or building of a required timeseries will start, if data can be built from the local history but they are not ready yet. The function will return the amount of data that will be ready by the moment of timeout expiration. "
The confusion I'm experiencing concerns the last sentence - i.e. what is the timeout expiration related to (i.e. is this the timeout of the CopyXXX() function or a timeout for downloading data or something different again?) and how does CopyXXX() know the amount of data that will be ready if it still requires downloading?
Also, I'm a bit confused as to why MetaQuotes chose to simply return a -1 if a CopyXXX() function is called within an Indicator and data requires downloading/assembling, while the equivalent of a CheckLoadHistory() function needs to be written (https://www.mql5.com/en/docs/series/timeseries_access#synchronized) and called every time a CopyXXX() function is called within an Expert or Script? Why not simply return a -1 within Experts and Scripts as well? Or have I got this completely wrong?
Any help on this would be greatly appreciated.