10 years 1 minute historical data for 15 currencies

To add comments, please log in or register
Eddy Taylor
75
Eddy Taylor  

Hi,

Does anyone know how to get/download 10 years 1 minute historical data for 15 currencies (and any other instrument) on MT4?

-For extensive backtesting

Phillip
803
Phillip  

Try these resources:

The metaquotes "download" data is really to be avoided at all costs in my opinion, loaded with spurious ticks, bad candles, and missing candle gaps all over the place.

The disktrading data goes back the farthest (~1998) but contains a significant percentage of contaminated ticks/candles (spurious ticks as well as your typical "indicative data" averaging artifacts), and will run you ~$135 for entire set of forex pairs.

The FXDD M1 data only goes back to ~2005 but it does come from a single price feed (FXDD's) so the quality is inherently higher.

The Forexite data is some of the best quality and goes back to 2001, except for the NZDJPY pair which only goes back to 2003.

The Gain Capital and Dukascopy datasets are a pain in the ass to retrieve and compile, I gave up on them once I found the disktrading/FXDD/Forexite resources but I included them here for sake of completeness as you may find them to be of value.

FourX
1316
FourX  
Eddy-2d wrote >>

Hi,

Does anyone know how to get/download 10 years 1 minute historical data for 15 currencies (and any other instrument) on MT4?

-For extensive backtesting

It's great idea, but most impractical.

I hope you have a supercomputers that you can dedicate specifically to each task or you will be old and grey before you get any usefull results trying to cover such magnititudess of data.

I'd also suggest that you have VERY good UPS/power conditioning as it is unlikley that you won't get some kind of irregularities / disruption in power before these finnish running: most likley when you are at ~ +95% finished.

Kepp us posted on the reality and fiesibilty of it and how, or if it works out and you are able to get any meaningfull results compelted.

Good luck.

Sarath
3
Sarath  

Is MT5s strategy tester is Multithreaded? I know MT4s is not, u can't take advantage of a multi core CPU.

Phillip
803
Phillip  

Regarding UPS:

I use inexpensive ($75) APC ups and works just fine, backtesting periods >1month continuous without issue. MT4 can be hibernated while actively running a backtest, so you set your APC to hibernate on extended power-outage (>5min in my setup) and then when I power up my computer once power has been restored the MT4 backtesting session continues unaffected.

Regarding lack of multi-threading in MT4 backtester:

For my quadcore rigs I simply duplicate the MT4 install folder (manually copy-and-paste) on my hard drive and launch as many simultaneous instances of MT4 as my CPU is capable of processing. Two instances for dual-core, four instances for quad, etc. Then I intentionally break-up the parameter ranges of the backtest to be done in each instance such that the aggregate output of all the backtests running in parallel results in the entire parameter space being sampled during backtesting.

For example say I want to backtest/optimize the averaging period for an RSI indicator spanning the values 3 to 200. I would setup the backtesting in MT4 Instance#1 to test RSI averaging period ranging 3-75, Instance#2 would backtest across values 76-125, Instance#3 would backtest across 126-165, and Instance#4 would backtest 166-200. (the backtested ranges are not identical in length because the shorter ranges like RSI(10) and RSI(20) are much faster to compute than the longer averaging periods like RSI(175) so you learn to manual weight the distributed load across your simultaneous MT4 instances and reduce the timegap between when the first and last backtesting runs complete)

I also setup my EA's to export tabulated results and then created an Excel file that automatically imports those four output files and culminates the results in one excel spreadsheet location for easier filtering/sorting/saving/etc.

If you are backtesting across multiple currency pairs it is equally easy to assign on currency pair backtest to each MT4 instance and run them in parallel. I use 5 quadcore rigs (so total of 20 cpu cores) to backtest/optimize 19 currency pairs simultaneously (no EA cross-currency dependencies obviously).

The only occasion where I wish MT4's backtester was multithreaded is that very very rare occasion where I find myself running just a single backtest run (no optimization) as a form of spotchecking (debug/etc). A minor nuisance really. The bigger issue is the lack of memory accessing >1GB per MT4 instance. Really puts an artificial ceiling on the depth of the historical data you can analyze (~6-7m candles in aggregate across all active currencies/charts in the specific MT4 instance).

To add comments, please log in or register