MT5 Data
I know that there has been a number of questions relating to the historical data on MT5 but none of them seem to have a good answer to solve the issue.....
My first issue:
My broker has had some problems with MT5 and it has frozen the feed to clients for upto 40 minutes at a time. Now there are gaps in the charts.
Before you say delete the history file, done that same gaps appear.
Yes I could ask my broker to update the gaps at there end, but honestly getting a broker to do that everytime there is an issue is not going to work is it.....
So come on Metatrader guys you get loads of money for this software how about setting up a proper reliable feed and maintaining a good historical base to get missing data from and altering MT5 so that it will get just the gap data?
My second issue:
MT5 supposedly builds all timeframe data from M1 data. But my M1 data only goes back a few months, whereas other timeframes go back many more and even years. This sounds like the old MT4 data setup to me which was to be blunt crap. So how is my MT5 getting this other timeframe data when it does not have the M1 data?
All the tools of MT5 become useless if the data is crap, and even allowing people to import there own data won't really help many, what we need is a single reliable history server to get all the missing M1 data.
Hi MaxTrader,
I think you should familiar yourself with MT5 before accusing something, because not all of us have the same problem like yours.
1. Open MT5 > click Tools > click Option (or - damn it - just press Ctrl + O) > select chart tab > on "Max bars in chart" select "Unlimited" .
2. Must to do, Open Monthy Chart and scroll down to the left side of your chart (make sure to disable Auto scroll - coz auto scroll is annoying). Keep Scroll down to the left as long as you want/need. It will download all M1 data that is needed to built that Monthly chart.
I only have EURUSD and it's about 194 MB from MetaQuotes server alone. Remember that Euro was formed in 1999 so my .hcc files become big starting from 1999.hcc, and it took so long to download and built this huge size of data, so be patient is a good idea. If you want to see where your .hcc is stored, just click "File" and select "Open Data Folder".
After you finish with Monthly chart, do the same with other TF, and it also will take sometime. However don't worry about this, when the next time you open other TF (other than M1 chart) it will be shown in no time.
3. If you have gap data, right click your chart and select Refresh. It will download M1 and repair that gap.
4. If those doesn't works, you may check your broker against another broker by adding servers beside your broker. I have MT5 from MetaQuotes, and I add several server from different broker. Open MT5 > Open Navigator (Ctrl + N) > right click account and select "Open an account". There you will have "Open an Account -Trading server" window. Double click on "add new server like access:metatrader5.com:443", and type in any broker name that have or may have MT5 server.
I actually just type in any broker name without really knowing whether the broker have MT5 server or not, and viola ... now I have 7 servers from 6 brokers ... LOL ... I just need one :).
5. If you still have chart freeze up thing and data gap, check against MT4, just in case MT4 is also freeze up and check your internet connection - usually internet connection is the problem, because like I said, other user (like me) may never experience your issues. It may good idea to have several MT running at the same time, to smooth the data streaming. I have 3 - 4 MT4 running and everything is OK with all MT4 and MT5.
6. Cheer up, it is annoying but it's not the end of the world. I don't have your issue so there's good chance you will free from this problems, ... however, after all I write here to help you, and if the problem still persist ... thank God it's only you who have these problems.
All I write there, are based on my experience with MT prior 655, now is MT 655 with MetaQuotes server.
:D
1. Unfortunately, the only way is to ask your broker. This can be fixed only on the broker side.
btw. deleting the history files completely is not necessary. It's enough t execute the "Refresh" command in the chart context menu. If the price data is corrected on the broker side, update will be applied the data on your side.
2. You're right about the M1 data. All timeframes are created on its basis. Probably, you limited the maximum bars that can be shown in a chart, and that is the reason why you see a few months of M1, and a few years on a weekly timeframe.
Just go to Options -> Charts and check the value of the "Max bars in chart" option
- www.mql5.com
Hi MaxTrader,
I think you should familiar yourself with MT5 before accusing something, because not all of us have the same problem like yours.
:D
Hi, okay I have read your comments but unfortunately it does not solve the issue as follows:
1. The gaps were definately caused by the broker - they agreed it was their server that was down. Refreshing my chart makes no difference gaps are still there, so by the time brokers have been running a couple of years with system glitches everybodies data will be full of holes again just like MT4 was/is.
2. My charts are set to unlimited and having done the above suggestions I can confirm that my brokers M1 histoyr goes only as far as March 2012 - how crap is that I tried Alpari Uk theirs goes back to 2010 I tried Metaquotes demo like you and it goes back further.
3. So it seems clear to me that different brokers are choosing to limit the amount of M1 data available which is crap and I don't understand why the software developers are allowing it to occur as it makes mt5 useless for anything other than trading.
4. It also does not answer the question of how caome you still get daily data etc going back further - where is the info coming from if not M1.
SO I still stand by suggestion that metaquotes should provide a reliable history server where gap data can be collected from before it is too late and everyone dismisses MT5 test results because of poor data just like on MT4.
thanks for your help I learnt a little more but metaquotes need to put more pressure on brokers to do the right thing.
iCustomChart indicator allows to create custom charts based on custom history files.
Is not free, but highly recommended.
- Konstantin Gruzdev
- www.mql5.com
Hi MaxTrader,
Thanks for the updates, that confirm what Alexx said (#1). I think the problem is at downloading cost. My EURUSD is about 194 MB goes back to 1999, for non-Euro pair that must be bigger than that. If there are 1,000 traders install 1 MT5, then there should be about 189 GB download from broker server for EURUSD alone - I know that's not big for server provider. However, how many are they if there are more than 1,000 traders install more than 1 MT5 and downloading more than 1 pair each month ?. I think downloading cost is what they want to avoid and so they limit the amount of M1 data.
Now that you brought this up, I do hope that MetaQuotes keeps their server data alive, so we can download more data back further time - especially since there are no means - at the moment - to create our own .hcc files, we certainly depend on what MetaQuotes provide with historical data then, and I do hope there are ways to create our own .hcc files also.
By the way, you may also want to use SymbolIsSynchronized (https://www.mql5.com/en/docs/marketinformation/symbolissynchronized) just to check if data is synched with broker's, though that does not tell whether our data has a hole and gap or not
I think all of us can create what FinGeR point at, however I prefer my data on it's own chart and not on some other pair chart. Thanks for the link FinGeR.
And thanks for the update MaxTrader.
:D
- www.mql5.com
@MaxTrader, I have found the same problem using Alpari. I posted about my problem here, and was redirected to your post.
It would seem that if a given broker decides not to store historical data before a certain date, then there is no way to get that data. As such, back-testing becomes impossible.
If I cannot back-test then I would posit that MetaTrader becomes useless to me. I am not going to spend times developing strategies in another platform's language, back-test them there, and then port that code over to MetaTrader to do the trading... I'm just going to do the trading on that other platform.
As such, I totally back your suggestion of a reliable history server.
At the moment it looks like I will have to look for another trading platform where this problem does not exist. This is very bad news for me, because I am a C++ programmer and as such was naturally keen to use MQL5 with it being so closely related to C++.
I would be very interested in hearing whether anyone has a solution to this.
@MaxTrader, I have found the same problem using Alpari. I posted about my problem here, and was redirected to your post.
It would seem that if a given broker decides not to store historical data before a certain date, then there is no way to get that data. As such, back-testing becomes impossible.
If I cannot back-test then I would posit that MetaTrader becomes useless to me. I am not going to spend times developing strategies in another platform's language, back-test them there, and then port that code over to MetaTrader to do the trading... I'm just going to do the trading on that other platform.
As such, I totally back your suggestion of a reliable history server.
At the moment it looks like I will have to look for another trading platform where this problem does not exist. This is very bad news for me, because I am a C++ programmer and as such was naturally keen to use MQL5 with it being so closely related to C++.
I would be very interested in hearing whether anyone has a solution to this.
Hi guys. This question is mostly to onewithzachy
I just started with this patform. I was using eSignal datafeeds 15 min delay for stocks on another platform(as apayd service). Does anyone knows how can I connect Metatrader5 to data feed provider of European stocks?
Hi guys. This question is mostly to onewithzachy
I just started with this patform. I was using eSignal datafeeds 15 min delay for stocks on another platform(as apayd service). Does anyone knows how can I connect Metatrader5 to data feed provider of European stocks?
1. Find a broker that trade European stocks, maybe you can find it here for a start http://www.forexpeacearmy.com/public/forex_broker_reviews and http://www.forexfactory.com/forumdisplay.php?f=74. The data can be use for backtesting.
2. Or using what FinGer mention above (https://www.mql5.com/en/forum/7028#comment_217505 - if you can, write the code yourself, coz I hate promoting commercial product in forum ;) ), but that way, the data can not be used for backtesting :(
- www.forexpeacearmy.com
I have done a fair amount of work to try and resolve the above issue.
As @onewithzachy suggested - it is possible to download data from different brokers. I created a script (details here) to download the minute bars individually from each broker and export them to file.
The filename format is "number of minute bars"-"currency pair"-"time period"-"broker"
- 4367984-GBPUSD-M1-AlpariFS-MT5.csv
- 4477443-GBPUSD-M1-MIGBank-Demo.csv
- 4676701-GBPUSD-M1-InstaForex-Server.csv
- 4745042-GBPUSD-M1-MetaQuotes-Demo.csv
- 670338-GBPUSD-M1-LiteForex-MT5.com.csv
- 88328-GBPUSD-M1-GoMarkets-Demo.csv
- 909085-GBPUSD-M1-AlpariUK-MT5.csv
- 917473-GBPUSD-M1-FXDD-Demo.csv
What a disparity - GoMarkets only has 88328 minute bars between now and 2000!
Anyway, I then parsed these files with a python script to do some sanity checking...
My first test was to look for gaps in time. Since I had requested minute-bars for the period of today back until 2000 I looked for any gap in time over 60 seconds long. If I found a gap which went over a weekend I subtracted the period of time for when the market is closed. This left me only with gaps which shouldn't have been gaps (ie: it pointed our missing data).
Sadly, every single broker had missing data. I created some graphs of the gaps (the x-axis is time, decreasing from now on the left until the year 2000 on the right; the y-axis is the length of the gap)
Even MetaQuotes-Demo, which has the largest number of minute bars, has huge gaps.
This is the gaps chart for GoMarkets-Demo:
This is the gaps chart for MetaQuotes-Demo, which has the most data of all the 'brokers':
As you can see there are gaps everywhere - the quality of data is quite appalling.
Anyway - I decided to press on... perhaps I could ignore the breaks in the data and do some other useful analysis. So I created a tick database:
The original idea was to have a record of each broker's ticks, and then calculate the average tick for each minute; following which I would be able to calculate the average standard deviation of each broker's data - thereby pointing to which offered the best spread etc.
Unfortunately I came up against a more serious problem... inserting the minute bars into the database violated my datetime_broker_currency constraint - this was that for a given currency pair and broker, the date/time should be unique. They weren't!!!
Several brokers had more than 1 set of OLHC for the same period of time! This is absolutely crazy - I really cannot believe that MetaQuotes would not use some kind of sanity check on its data to ensure it follows a certain set of criteria.
Let me illustrate with one simple point in time:
$ grep '2010.01.04 11:57' Files/* Files/4367984-GBPUSD-M1-AlpariFS-MT5.csv:2010.01.04 11:57,1.61361000,1.61360000,1.61379000,1.61377000 Files/4367984-GBPUSD-M1-AlpariFS-MT5.csv:2010.01.04 11:57,1.61750000,1.61746000,1.61778000,1.61769000 Files/4477443-GBPUSD-M1-MIGBank-Demo.csv:2010.01.04 11:57,1.61751000,1.61751000,1.61778000,1.61770000 Files/4477443-GBPUSD-M1-MIGBank-Demo.csv:2010.01.04 11:57,1.60664000,1.60577000,1.60678000,1.60618000 Files/4477443-GBPUSD-M1-MIGBank-Demo.csv:2010.01.04 11:57,1.61751000,1.61751000,1.61778000,1.61770000 Files/4676701-GBPUSD-M1-InstaForex-Server.csv:2010.01.04 11:57,1.61750000,1.61750000,1.61780000,1.61770000 Files/4745042-GBPUSD-M1-MetaQuotes-Demo.csv:2010.01.04 11:57,1.61751000,1.61751000,1.61778000,1.61770000
Wow - AlpariFS has 2 entries for 4th January 2010 whilst MIGBank has 3 entries!
If I can jump to conclusions here I suspect what has happened is that both AlpariFS and MIGBank have loaded the MetaQuotes-Demo historical data on top of theirs (you can see that 1 of AlpariFS's ticks and 2 of MIGBank's ticks match the MetaQuotes-Demo tick).
The fact that MetaTrader allows duplicate dates is so dodgy - and it really calls into question the reliability of their system. It is for this reason that I have decided to move to another platform.
I hope this thread helps others others evaluate the quality of the back testing available in MetaTrader5.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
I know that there has been a number of questions relating to the historical data on MT5 but none of them seem to have a good answer to solve the issue.....
My first issue:
My broker has had some problems with MT5 and it has frozen the feed to clients for upto 40 minutes at a time. Now there are gaps in the charts.
Before you say delete the history file, done that same gaps appear.
Yes I could ask my broker to update the gaps at there end, but honestly getting a broker to do that everytime there is an issue is not going to work is it.....
So come on Metatrader guys you get loads of money for this software how about setting up a proper reliable feed and maintaining a good historical base to get missing data from and altering MT5 so that it will get just the gap data?
My second issue:
MT5 supposedly builds all timeframe data from M1 data. But my M1 data only goes back a few months, whereas other timeframes go back many more and even years. This sounds like the old MT4 data setup to me which was to be blunt crap. So how is my MT5 getting this other timeframe data when it does not have the M1 data?
All the tools of MT5 become useless if the data is crap, and even allowing people to import there own data won't really help many, what we need is a single reliable history server to get all the missing M1 data.