Slow backtest and historical data
Yes, using real tick data is much slower, but reason depends on both issues. Your code is usually the one with the most impact on the testing speed.
The usual problems are due to code doing things on every tick instead of only when a new bar forms.
The more efficient your code the better.
The external harddrive issue also has an impact obviously. So using an internal drive or an SSD will definitely help improve the speed of tests.
Yes, using real tick data is much slower, but reason depends on both issues. Your code is usually the one with the most impact on the testing speed.
The usual problems are due to code doing things on every tick instead of only when a new bar forms.
The more efficient your code the better.
The external harddrive issue also has an impact obviously. So using an internal drive or an SSD will definitely help improve the speed of tests.
Actually something happened to me that I found strange but it certainly depends on what you told me: with the historical data at 90% I had not inserted any function of this type that operates only at the opening of a new bar instead of every tick, but everything worked normally.
When I tried with this data at 99% instead I got the error array out of range. I inserted the function to execute only once when the bar was opened and everything was working again.
How can this depend on the historical data used?
For the rest in addition to the code I understood, I imagined it could also center the hd , surely I have to fall back on a better product.
Thank you so much for the quick and kind reply.
Actually something happened to me that I found strange but it certainly depends on what you told me: with the historical data at 90% I had not inserted any function of this type that operates only at the opening of a new bar instead of every tick, but everything worked normally.
When I tried with this data at 99% instead I got the error array out of range. I inserted the function to execute only once when the bar was opened and everything was working again.
How can this depend on the historical data used?
For the rest in addition to the code I understood, I imagined it could also center the hd , surely I have to fall back on a better product.
If you are getting array out of range then your code is faulty. It is probably not checking the number of available bars or something like that.
If you are getting array out of range then your code is faulty. It is probably not checking the number of available bars or something like that.
I ask to understand, not because I think there is no mistake on code.
Because FXT files generated by the terminal always generates extra 1000 bars before the test start date, but when you use 3rd party tools for generating FXT files with tick data, you the user may forget about this necessity and probably only generate data from the start date you plan to use.
As I said, the problem is your code, because you are probably not doing proper checks into indexing of bars and indicators. No matter the reason, you need to fix you code anyway.
Because FXT files generated by the terminal always generates extra 1000 bars before the test start date, but when you use 3rd party tools for generating FXT files with tick data, you the user may forget about this necessity and probably only generate data from the start date you plan to use.
As I said, the problem is your code, because you are probably not doing proper checks into indexing of bars and indicators. No matter the reason, you need to fix you code anyway.
I was amazed by the difference in the result between 90 and 99% in terms of backtesting.
I am now equipping myself with an SSD to improve speed. Thanks for your suggestions
Yes, using real tick data is much slower, but reason depends on both issues. Your code is usually the one with the most impact on the testing speed.
The usual problems are due to code doing things on every tick instead of only when a new bar forms.
The more efficient your code the better.
The external harddrive issue also has an impact obviously. So using an internal drive or an SSD will definitely help improve the speed of tests.
Now I understand that the optimization of the code is very important and must be written immediately in an appropriate way!
It is very good that you have understood that so quickly. Very few have discovered that. Well done!
It is very good that you have understood that so quickly. Very few have discovered that. Well done!
But it is normal that I tried to create an expert advisor that breaks out of the daily High and Low levels (it is an example), and while everything works perfectly on Alpari with historical data at 90% (I also inserted a Comment to see the prices and everything goes smoothly) on the historical data at 99%, the expert does not work as it should? That is, the comment does not give me the prices and does not enter the market as it should.
However, this only happens if I try to take the prices from another timeframe, if I do a test with expert advisors who do not have this particularity, everything seems to be fine ( I guess ).
When your EA requires data from a different time-frame or a different symbol, you have to carry out extra operations to make sure that the data has been retrieved and synchronised properly. The data is not explicitly available for you the moment the tester begins. You have to check for delays and errors.
Forum on trading, automated trading systems and testing trading strategies
Retrieving current history data from symbol that doesn't have an open chart on a particular period
Fernando Carreiro, 2022.04.25 11:20
In Multi-Symbol/Multi-Time-frame Indicators and/or EAs, you have to properly handle the synchronisation of the requested of data on a timely fashion by handling 4066/4073 errors. Failure to do this will cause it not to work properly.
Forum on trading, automated trading systems and testing trading strategies
Minimum requirements for showing different symbol info in-time info in one chart?
William Roeder, 2021.08.27 17:06
On MT4: Unless the current chart is that specific symbol(s)/TF(s) referenced, you must handle 4066/4073 errors before accessing candle/indicator values.
Download history in MQL4 EA - Forex Calendar - MQL4 programming forum - Page 3 #26.4 (2019.05.20)On MT5: Unless the current chart is that specific pair/TF, you must synchronize the terminal Data from the Server before accessing candle/indicator values.
Error 4806 while using CopyBuffer() - Expert Advisors and Automated Trading - MQL5 programming forum #10 (2020.12.15)
Is it mystical?! It is! - Withdraw - Technical Indicators - MQL5 programming forum (2019.05.31)
Timeseries and Indicators Access / Data Access - Reference on algorithmic/automated trading language for MetaTrader 5
Synchronize Server Data with Terminal Data - Symbols - General - MQL5 programming forum #2 (2018.07.17)
SymbolInfoInteger doesn't work - Symbols - General - MQL5 programming forum (2019.09.03)
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
In the end I had to install the MT4 in portable version (I found the guides here on the forum and everything went well and it is working) and I downloaded everything including the historical data on an external hard drive because my internal does not have enough space.
The question is this: it really takes a long time to do a backtest compared to before, let's talk that it took 2 hours longer for the same expert. Is this normal and may it depend on my code or is it because I am using an external hard drive which may not be fast enough?