Compound History Data Script

New comment
 
I came upon a realization today that history data are not all created equal. Data from some brokers may be more complete than others while other data still may contain many time gaps for which price data is not defined. This is very disconcerting. I downloaded a second copy of metatrader from the metaquotes website along with all available history from 'alpari' to find fairly significant gaps in the price data. Comparing these side by side using my brokers copy of metatrader &it's price data I found that it was complete where the alpari data wasnt but had gaps in other places! The disconcerning part came when I ran my already 'optimized' EA using both history data sets with the same parameter settings, on the exact same timeframe, currency, and test period and having it yield significantly different results. One showed a nice overall profit while the other showed a loss. Looking back at the data (after running backtest again in visual mode) the discrepancy very likely results from trades that subtend gaps in the data. Clearly a trade opened on one side of a 900 pip price gap and close on other, with a month's time having disappeared over a single 5 minute candle, can turn a good backtest into junk data. There must be some way to alleviate this problem with history data. After all, the data is out there..a 'correct' accout of price data is stored at least between the many version stored on various servers. What if several price data sets can be consolidated into one 'complete' set, where a gap in one data set is filled in by present data for the same time period in another. I imagine this can be done using a script that reads a data set, while noting where time has disappeared then reading the next data set and if price data for the time where data was missing in the first exists in the second, it appends that data into that time period all the while writing a new .hst file? What do you think? Any suggestions to get me started? Any issues I may encounter? Anyone wanna do it for me? :)
 

Generally, the larger the time-frame the expert trades, the less it would be sensitive to different price feeds. For smaller time-frames, the usual recommendation on the forum is to collect data from the broker u intend to trade with (from his Live account). This is not as easy as it sounds and sometimes not really practical.

There have been many discussions on the subject. I suggest you search first. One good thread comes to mind -> https://www.mql5.com/en/forum/118275. Also u might ask 1005phillip as he seems to have a good grasp on the subject, specifically u should read his posts in the thread I mentioned.

 
thanks gordon, that helped alot ..saved me alot of trouble too!
New comment
Reason: