Backtester skipping over blocks

[Metal10k]

Ive been working with mql5 code for a while now, appreciating the wonders of the object oriented world :) however I have hit a brick wall.

Basically I have coded an algorithm (highly computationally intensive) which all classes have been tested and debugged using scripts. The functions execute flawlessly in scripts, and even in an expert as they were intended on a live chart. However, when it comes to backtesting I noticed that the results of the algorithm are garbage, and the expert is thus not functioning as intended. As there is no real way to debug an expert inside of the backtester as it iterates through the history (which I might add would be extremely useful), I have had to resort to the old fashioned memory buffer printing to 'debug' the code within the backtester.

The only conclusion that I can give from this is that the backtester is basically skipping over whole chunks of code if it deems them to be taking too long. Is there some kind of inherent failsafe (id assume to prevent dodgey code iterating forever and to speed up the backtesting process) which basically skips out of any loops if a certain time is exceeded? If so is there any possible way to override this? If not this is a huge problem for anyone who is serious about coding (and of course backtesting) advanced experts which crunch through a lot of data. OOP really opens the door for advanced learning algorithms etc, but itd obviously be necessary to be able to backtest them for this new functionality to be of serious use. I am hoping though that the slowed down RT Visual Mode when it is implemented may provide a workaround for this. 

For obvious reasons I'm not going to share my code, however if its really necessary I am confident I can recreate the problem using a simple iterative example (a few nested for loops which iterate for a long period of time on init and print stuff out).

 

Any help would be greatly appreciated.

Alex 

[Rashid Umarov]

Have you read Help of client terminal MetaTrader 5?

I think you didn't pay your attention to this section

[Metal10k]

Rosh:

Have you read Help of client terminal MetaTrader 5?

I think you didn't pay your attention to this section

Hello, thanks for your quick response. Yes I have looked at the log and it contains the same gaps in the iterations as the log inside mt5. Unless there is an option for truncation, I dont think it is processing the data.

 

to clarify class has a loop something like this:

for (int i = 0; i<1000; i++)

Print("i = ", i, "    ",  somefunc());

when I run the class with this init function on a fresh live chart, the log (both internal and external) begins at 0 and iterates nicely through everything. The init function whence calling this class starts at something like 16 and finishes at 70, missing out a chunk of intermediary dumps outside of the loop also. Is there definately no way the tester can 'skip' any of the mql5 code? However I'm happy to accept the logger isnt printing everything for now. Whats causing the problems though is this:

I seem to have another issue with copying the buffer out of an indicator in my class too.  Functions called in this order:

 

 A_IndicatorHandle = iRSI(symbol, _Period, someparam, PRICE_CLOSE); 

ArraySetAsSeries(A_IndicatorBuffer,true); 

CopyBuffer(A_IndicatorHandle,0,0,Backtest.window+1,A_IndicatorBuffer); 

dosomething(A_IndicatorBuffer); 

 

When this sequence is run on a live chart (called inside init or whatever, the handle is generated, the buffer is copied and populated correctly. On the backtester however, the handle is generated correctly, but the copybuffer command returns -1( data outside TERMINAL_MAXBARS) . The class is in an external header file, could this somehow influence the interperatation of index 0 as opposed to backtester (the current timestep)?

[Rashid Umarov]

Metal10k:

Hello, thanks for your quick response. Yes I have looked at the log and it contains the same gaps in the iterations as the log inside mt5. Unless there is an option for truncation, I dont think it is processing the data.

See agent log file. This file specified in the Tester Log

2010.06.28 19:32:51    Core 1    Log file "D:\MT5\Tester\Agent-127.0.0.1-3000\logs\20100628.log" written

[Metal10k]

Rosh:

See agent log file. This file specified in the Tester Log

2010.06.28 19:32:51    Core 1    Log file "D:\MT5\Tester\Agent-127.0.0.1-3000\logs\20100628.log" written

Ahh i see it now, brilliant thanks, that resolves the first problem. Any idea what might be wrong with the indicator?

 

I understand there can be a lag between the indicator initialising and populating its buffers

(been reading here https://www.mql5.com/en/docs/series/copybuffer)

Writing a routine like this causes a crash however: 

 

         ArraySetAsSeries(A_IndicatorBuffer,true);

         int copy;

            while(copy==-1)

               {

               copy = CopyBuffer(A_IndicatorHandle,0,0,window,A_IndicatorBuffer);

               Print("Copy = ", copy);

               } 

[Metal10k]

Whey, adding  Sleep(10000); seems to have fixed the problem. I guess the indicator needs some time to populate its buffers. Is there any way to check for this? I'm guessing that each computer will need a different sleep time so this is a bit of a fudge.  Adding sleep to a while loop seems to cause problems. Il experiment with this anyway.

[Metal10k]

         int copy, i=0;

         copy = CopyBuffer(A_IndicatorHandle,0,0,window,A_IndicatorBuffer);

          while(copy==-1 || i <1000)

             {

               Sleep(20);

               copy = CopyBuffer(A_IndicatorHandle,0,0,Backtest.window+1,A_IndicatorBuffer);

i++; 

             }

 

works for now

[Rashid Umarov]

Metal10k:

 

I understand there can be a lag between the indicator initialising and populating its buffers

(been reading here https://www.mql5.com/en/docs/series/copybuffer)

You certainly don't like reading. https://www.mql5.com/en/docs/series

Access to indicator and timeseries data is implemented irrespective of the fact whether the requested data are ready (the so called asynchronous access). This is critically important for the calculation of custom indicator, so if there are no data, functions of Copy...() type immediately return an error. However, when accessing form Expert Advisors and scripts, several attempts to receive data are made in a small pause, which is aimed at providing some time necessary to download required timeseries of to calculate indicator values.

The Organizing Data Access section describes details of receiving, storing and requesting price data in the MetaTrader 5 client terminal.