is there any known issue with CheckLoadHistory() ?

New comment
 

Dear fellows

I am having hard time with CheckLoadHistory() of article https://www.mql5.com/en/docs/series/timeseries_access and seeking guidance here if any known issues with it or I am doing something wrong?

I am using following code to sync data and when I start the MQL Platform the indicator hangs out and I have to delete it. This causes error[-3] stopped by used.

//+-----------------------------------------------------------------------------------------------------------------------------+
//| METHOD:       OnTick()
//| APPLICATION:  
//+-----------------------------------------------------------------------------------------------------------------------------+
void CSessionVPBase::OnTick(void) {

                string vMethod = "[" + mSymbol + "," + TimeToString(TimeCurrent()) + "] " + __FUNCTION__;
        //+---------------------------------------------------------------------------------------------------------------------------+
        //| checkLoadHisory() returns 0,1 or 2, if bars are synchronized with history, hence used >= 0
        //+---------------------------------------------------------------------------------------------------------------------------+
                int barCount = (int)(mMaxSessionCalculate/3) * 24;                                                                      // PERIOD_H1 Bars in mMaxSessionCalculate / No of Daily Session
                ResetLastError();
                datetime syncH1From = iTime(mSymbol,PERIOD_H1,barCount);
                int errH1 = GetLastError();
                ResetLastError();
                datetime syncM1From = iTime(mSymbol,PERIOD_M1,barCount*60);                                     // H1 barCount convert into M1 barCount
                int errM1 = GetLastError();

                int              resultH1       = cATS.checkLoadHistory(PERIOD_H1,syncH1From);
                int              resultM1       = cATS.checkLoadHistory(PERIOD_M1,syncM1From);

                if(resultM1 < 0 || resultH1 < 0) {
                        //CSessionVPBase::OnTick: barCount[792] syncM1From[1970.01.01 00:00]errM1[4401] syncH1From[1970.01.01 00:00]errH1[4401]
                        //CSessionVPBase::OnTick: BarsM1[0] >= MaxBarsM1[60000] TimeSeries Data not synchronized resultM1[-3] resultH1[-3], try on next Tick
                        PrintFormat("%s: barCount[%i] syncM1From[%s]errM1[%i] syncH1From[%s]errH1[%i]",vMethod,barCount,TimeToString(syncM1From),errM1,TimeToString(syncH1From),errH1);
                        PrintFormat("%s: BarsM1[%i] >= MaxBarsM1[%i] TimeSeries Data not synchronized resultM1[%i] resultH1[%i], try on next Tick",vMethod,Bars(mSymbol,PERIOD_M1),TerminalInfoInteger(TERMINAL_MAXBARS),resultM1,resultH1);
                        return;
                }
                //PrintFormat("%s: Bars[%i] >= MaxBars[%i] TimeSeries Data synchronized resultM1[%i] resultH1[%i] resultSTF[%i]",vMethod,Bars(mSymbol,PERIOD_M1),TerminalInfoInteger(TERMINAL_MAXBARS),resultM1,resultH1,resultSTF);

When I attach indicate back and recompile the indicator, it starts working.

I have used following checkLoadHistory() with little bit modification to use it in a class (cATS) where class is initialized with mSymbol.

//+-----------------------------------------------------------------------------------------------------------------------------+
//| METHOD:     checkLoadHistory()
//+-----------------------------------------------------------------------------------------------------------------------------+
int CAccessTimeseries::checkLoadHistory(ENUM_TIMEFRAMES pChartTF,datetime pStartDateTime) {

                string vMethod = "[" + mSymbol + "," + EnumToString(pChartTF) + "] " + __FUNCTION__;

                datetime first_date = 0;
                datetime times[100];
                //--- check pChartTF
                if(pChartTF == PERIOD_CURRENT)     pChartTF = Period();
                //--- check if symbol is selected in the Market Watch
                if(!SymbolInfoInteger(mSymbol,SYMBOL_SELECT)) {
                        if(GetLastError() == ERR_MARKET_UNKNOWN_SYMBOL)                                                                         return(-1);
                        SymbolSelect(mSymbol,true);
                }

                //--- check if data is present
                SeriesInfoInteger(mSymbol,pChartTF,SERIES_FIRSTDATE,first_date);
                if(first_date > 0 && first_date <= pStartDateTime)                                                                              return(1);

                //--- don't ask for load of its own data if it is an indicator
                if(MQL5InfoInteger(MQL5_PROGRAM_TYPE) == PROGRAM_INDICATOR && Period() == pChartTF && Symbol() == mSymbol)                      return(-4);

                //--- second attempt
                if(SeriesInfoInteger(mSymbol,PERIOD_M1,SERIES_TERMINAL_FIRSTDATE,first_date)) {
                        //--- there is loaded data to build timeseries
                        if(first_date > 0) {
                                //--- force timeseries build
                                CopyTime(mSymbol,pChartTF,first_date+PeriodSeconds(pChartTF),1,times);
                                //--- check date
                                if(SeriesInfoInteger(mSymbol,pChartTF,SERIES_FIRSTDATE,first_date))
                                if(first_date > 0 && first_date <= pStartDateTime)                                                              return(2);
                        }
                }

                //--- max bars in chart from terminal options
                int max_bars = TerminalInfoInteger(TERMINAL_MAXBARS);
                //--- load mSymbol history info
                datetime first_server_date = 0;
                while(!SeriesInfoInteger(mSymbol,PERIOD_M1,SERIES_SERVER_FIRSTDATE,first_server_date) && !IsStopped())
                        Sleep(5);

                //--- fix start date for loading
                if(first_server_date > pStartDateTime)                                                                                  pStartDateTime = first_server_date;
                if(first_date > 0 && first_date < first_server_date)
                        Print("Warning: first server date ",first_server_date," for ",mSymbol," does not match to first series date ",first_date);

                //--- load data step by step
                int fail_cnt = 0;
                while(!IsStopped()) {
                        //--- 1.wait till timeseries re-build process is over
                        while(!SeriesInfoInteger(mSymbol,pChartTF,SERIES_SYNCHRONIZED) && !IsStopped())
                                Sleep(5);
                        //--- 2.request how many bars we have
                        int bars = Bars(mSymbol,pChartTF);
                        if(bars > 0) {
                                //PrintFormat("%s: bars[%i] > 0 and max_bars[%i]",vMethod,bars,max_bars);
                                if(bars >= max_bars)                                                            return(-2);     // bars more than ones that can be drawn in the chart, exit
                                //--- ask for first date
                                if(SeriesInfoInteger(mSymbol,pChartTF,SERIES_FIRSTDATE,first_date))
                                        if(first_date > 0 && first_date <= pStartDateTime)                      return(0);
                        }

                        //--- copying of next part forces data loading
                        int copied = CopyTime(mSymbol,pChartTF,bars,100,times);
                        if(copied > 0) {
                                //--- check for data
                                if(times[0] <= pStartDateTime)	return(0);
                                if(bars+copied >= max_bars)	return(-2);
                                fail_cnt = 0;
                        }
                        else {
                                //--- no more than 100 failed attempts
                                fail_cnt++;
                                if(fail_cnt >= 100) return(-5);
                                Sleep(10);
                        }
                }
                //--- stopped by User
                PrintFormat("%s: mSymbol[%s]",vMethod,mSymbol);
                return(-3);

} // End of method checkLoadHistory()
Documentation on MQL5: Timeseries and Indicators Access / Organizing Data Access
Documentation on MQL5: Timeseries and Indicators Access / Organizing Data Access
  • www.mql5.com
Organizing Data Access - Timeseries and Indicators Access - MQL5 Reference - Reference on algorithmic/automated trading language for MetaTrader 5
 
Anil Varma:

Dear fellows

I am having hard time with CheckLoadHistory() of article https://www.mql5.com/en/docs/series/timeseries_access and seeking guidance here if any known issues with it or I am doing something wrong?

I am using following code to sync data and when I start the MQL Platform the indicator hangs out and I have to delete it. This causes error[-3] stopped by used.

When I attach indicate back and recompile the indicator, it starts working.

I have used following checkLoadHistory() with little bit modification to use it in a class (cATS) where class is initialized with mSymbol.

When/where in the indicator are you calling OnTick() ?

Could you provide a compilable version of the code, so it is possible to reproduce the issue?
 
Dominik Egert #:
When/where in the indicator are you calling OnTick() ?

Could you provide a compilable version of the code, so it is possible to reproduce the issue?

Dear @Dominik Egert

Thanks a lot for your reply and support to bail me out of the error issue.

Attached is compliable version in Zip format.

This indicator is really beyond my current coding capabilities and helping me a lot in learning MQL much better way. As I am still having other challenges in the code, please ignore areas not related and you can focus only on CSessionVPBase::OnTick(void) method at line 259 and if(resultM1 < 0 || resultH1 < 0) error there.

In addition any suggestion to improvise coding style will always be welcomed.

Thanks again a lot and look forward for solution.

Files:
Compilable_iSessionVP.zip  48 kb
 
Anil Varma #:

Dear @Dominik Egert

Thanks a lot for your reply and support to bail me out of the error issue.

Attached is compliable version in Zip format.

This indicator is really beyond my current coding capabilities and helping me a lot in learning MQL much better way. As I am still having other challenges in the code, please ignore areas not related and you can focus only on CSessionVPBase::OnTick(void) method at line 259 and if(resultM1 < 0 || resultH1 < 0) error there.

In addition any suggestion to improvise coding style will always be welcomed.

Thanks again a lot and look forward for solution.

I will be able to check your code earliest on Thursday... Maybe someone else finds it challenging to check in the meantime.

Shouldn't be to hard to figure out the issue, I guess... But we will see.
 
Anil Varma #:

Dear @Dominik Egert

Thanks a lot for your reply and support to bail me out of the error issue.

Attached is compliable version in Zip format.

This indicator is really beyond my current coding capabilities and helping me a lot in learning MQL much better way. As I am still having other challenges in the code, please ignore areas not related and you can focus only on CSessionVPBase::OnTick(void) method at line 259 and if(resultM1 < 0 || resultH1 < 0) error there.

In addition any suggestion to improvise coding style will always be welcomed.

Thanks again a lot and look forward for solution.

So, I have analyzed it to some extend. Here is what I have found:

DealingWithTime.mqh

In line 235 to 238 there is a "Sleep()" command. - It is ineffective in indicators.

In line 633 there is an undocumented function call to MQL5InfoInteger, should probably be MQLInfoInteger.

In line 636 the error 4401 is raised for "ERR_HISTORY_NOT_FOUND".

In line 651, if that call fails, a while loop will run again using "Sleep" with no effect in indicators.

In line 663 the while loop will never turn false, keeping it running forever, because the chart thread is blocked by the indicator. - this results in a "race condition": One is waiting for the other while the other will never satisfy the condition, because (in this case) it will never get the chance to download the data.


This is the actual issue with the code. - In general, I would say the whole code needs a bit more error handling, and awareness of what it is actually doing. I have gotten multiple different error codes on the way, and the code should be written in such way, that it takes care of all error codes in one way or another.


So basically, the coe is incompatible to indicators, as it is at the moment.
 
So basically, the code is incompatible to indicators, as it is at the moment.

Hi @Dominik Egert Thanks a lot for your support and taking time off to help me out.

In line 235 to 238 there is a "Sleep()" command. - It is ineffective in indicators. Is there any harm, if still included in code and there is a call from EA or Strategy Class, so it to work?

Somewhere in the documentation I found ...

Access to indicator and timeseries data is implemented irrespective of the fact whether the requested data are ready (the so called asynchronous access). This is critically important for the calculation of custom indicator, so if there are no data,

functions of Copy...() type immediately return an error. However, when accessing from Expert Advisors and scripts, several attempts to receive data are made in a small pause, which is aimed at providing some time necessary to download required

timeseries or to calculate indicator values.

Can you guide me, if an Indicator is calling an OOP class (not an EA or Script) and Copy...() type is located in such OOP class. Call to this Copy...() will be treated as from an Indicator OR from an EA/Script?

and hence if call is treated from an Indicator, your conclusion (So basically, the code is incompatible to indicators, as it is at the moment.) should held true.

I am trying to digest other clues from you and see any work around.

One more request, if you are aware of another article which handles such Organized Data Access for Indicators, you can point me to it.

I am also closing this thread with conclusion that checkLoadHistory() of article https://www.mql5.com/en/docs/series/timeseries_access is incompatible to indicators, as it is at the moment. I would like to request  MQL Service Desk (@MQL-Services), to look into this and if possible provide and updated version.

Thanks a lot again and regards.

Documentation on MQL5: Timeseries and Indicators Access / Organizing Data Access
Documentation on MQL5: Timeseries and Indicators Access / Organizing Data Access
  • www.mql5.com
Organizing Data Access - Timeseries and Indicators Access - MQL5 Reference - Reference on algorithmic/automated trading language for MetaTrader 5
 
Anil Varma #:

Hi @Dominik Egert Thanks a lot for your support and taking time off to help me out.

In line 235 to 238 there is a "Sleep()" command. - It is ineffective in indicators. Is there any harm, if still included in code and there is a call from EA or Strategy Class, so it to work?

Somewhere in the documentation I found ...

Access to indicator and timeseries data is implemented irrespective of the fact whether the requested data are ready (the so called asynchronous access). This is critically important for the calculation of custom indicator, so if there are no data,

functions of Copy...() type immediately return an error. However, when accessing from Expert Advisors and scripts, several attempts to receive data are made in a small pause, which is aimed at providing some time necessary to download required

timeseries or to calculate indicator values.

Can you guide me, if an Indicator is calling an OOP class (not an EA or Script) and Copy...() type is located in such OOP class. Call to this Copy...() will be treated as from an Indicator OR from an EA/Script?

and hence if call is treated from an Indicator, your conclusion (So basically, the code is incompatible to indicators, as it is at the moment.) should held true.

I am trying to digest other clues from you and see any work around.

One more request, if you are aware of another article which handles such Organized Data Access for Indicators, you can point me to it.

I am also closing this thread with conclusion that checkLoadHistory() of article https://www.mql5.com/en/docs/series/timeseries_access is incompatible to indicators, as it is at the moment. I would like to request  MQL Service Desk (@MQL-Services), to look into this and if possible provide and updated version.

Thanks a lot again and regards.

In indicators, actually the OnCalculate function is a passive part of the copy and download process itself.

You should orient yourself around the delivered data you get in the parameters from OnCalculate.

If you code an indicator, no matter what you name your functions, they are part of the indicator and therefore bound to the restrictions of that program.
 
Dominik Egert #:
In indicators, actually the OnCalculate function is a passive part of the copy and download process itself.

You should orient yourself around the delivered data you get in the parameters from OnCalculate.

If you code an indicator, no matter what you name your functions, they are part of the indicator and therefore bound to the restrictions of that program.

Thanks for clarifications :)

 

Forum on trading, automated trading systems and testing trading strategies

Help: array out of range, though in do...while loop I have tried to check error and exit

Alain Verleyen, 2024.01.31 16:09

You need to understand and deal, with the asynchronous nature of the data functions.

Using loop or sleep is a bad approach. When a data function returns -1 it (usually) means the data are not ready. The best approach is then to use events, either waiting the next tick, or forcing some events with a timer or custom event.

CheckLoadHistory() is useless, there is no need for an updated version. It could maybe be useful the first time you launch a program to collect data (but it's done automatically according to the requested data). Anyway, on indicators, which are using data from different timeframe or symbol as the current chart, EACH TIME you request data, it's ALWAYS possible the data are not ready. You NEED to deal with it, there is no shortcut, not miracle solution.

And I repeat : in indicators "Using loop or sleep is a bad approach".

 
Alain Verleyen #:
You NEED to deal with it, there is no shortcut, not miracle solution.

Hi @Alain Verleyen

Your message is loud and clear, I understood it very well. However, "You NEED to deal with it, there is no shortcut, not miracle solution." is where I need some guidelines as how to deal with it. Potential solution I found and thought would work was CheckLoadHistory(), but it has been proved to not useful.

I will try again tomorrow with fresh mind, now I am exhausted from whole days coding work.

Thanks for you suggestions.

 
Anil Varma #:

Hi @Alain Verleyen

Your message is loud and clear, I understood it very well. However, "You NEED to deal with it, there is no shortcut, not miracle solution." is where I need some guidelines as how to deal with it. Potential solution I found and thought would work was CheckLoadHistory(), but it has been proved to not useful.

I will try again tomorrow with fresh mind, now I am exhausted from whole days coding work.

Thanks for you suggestions.

If I may throw in some grains of salt to this.

OnCalculate is called frequently. If you cannot receive the data on this call, you will have placed a requests for the data, return from OnCalculate and wait for the next call.

When returning from OnCalculate, always remember to return your current progress on calculations for your indicator.

So if it is the first run, and you request data, but you don't get it. You simply return 0. This way on the next call, you can retry to receive the data, if successful, start your calculation loop for the history data.

Whenever you are not done, but meet an update by the terminal, return your current state, and continue in the next call with where you were in your last call.
12
New comment
Reason: