MT5 : ticks data questions, issues and bugs. All about ticks. - page 3

 
Arthur Albano:

Instrument: WDOZ18. Broker M:

Broker R doesn't have ticks after market close:


At 18:00 (theoretical(?) market close):

Broker M:



Broker R:

Broker R fails Boyce-Codd, or it's simply not displaying ticket number (primary key), for instance. Ticks report does not have column display options.

What a mess from these brokers and MT5 for a supposed centralized market !!!
 
Alain Verleyen:
What a mess from these brokers and MT5 for a supposed centralized market !!!
What I've learned from MT5: no EA is reliable if sl or tp are smaller than (high-low) tested on OHLC. To use tick for backtesting: careful filtering and step-by-step checking, among other things. :P
 
Fernando Carreiro:

Something tells me you will never get a satisfactory answer to this! Whether it is a bug or broker manipulation, neither party is going to address the issue because it is not in their best interest to correct it.

Probably the only option available to you will be to adjust your strategy or your code to cope with it in the best way you can, hoping for the best but planning for the worst!

One observation that is in your favour, however, is the fact that there were more ticks in the historical data than the number of ticks counted in the OHLC tick volume. Had it been the other way round, then it would really have been a severe problem.

Another point which you can investigate is to harvest tick data in real-time for a few days or even weeks and then compare it to historic tick data (by clearing bases and re-downloading it again), to see what discrepancies there are between the two, and then adjust your project to work around that issue should there be problems with this.

Harvesting from the book is a wise choice, but requires setting up a DB to store all data.
 

So, I got some interesting information from Slava, one of the main developers from Metaquotes.

So the bar (OHLC) ticks volumes is actually counting ticks when Bid/Ask both change and when Bid alone change, but NOT when Ask alone is changing.

After checking the data from post #1 with this information, effectively I got very close values for Bar ticks volume and ticks data count. There are still some little differences though.

Forum on trading, automated trading systems and testing trading strategies

New version of MetaTrader 5 build 1930: Floating graphs and .Net libraries in MQL5

Alain Verleyen , 2018.11.30 19:16

Thank you.

Closer, but still some differences.

CopyTicksRange (EURUSD, MN1) TICKS: PERIOD_MN1 : from 2018.10.01 00:00 to 2018.11.01 00:00, TICKS_BIDASK = 2920555 (OHLC Tick volume = 2448455 )
CopyTicksRange (EURUSD, MN1) TICKS: Bid / Ask = 1790166, Bid = 642184, Ask = 477455, Duplicate (Bid / Ask) = 10750 Total = 2432350 (Bid + Bid / Ask ticks) / 2909805 (real: all, no duplicate ) / 2920555


 

Forum on trading, automated trading systems and testing trading strategies

MT5 : Issues and bugs working with real ticks/custom ticks.

Alain Verleyen, 2018.12.07 05:07

Beta build 1958 (and previous from ?).

While using CustomTicksReplace() to import a 10 millions ticks buffer on a custom symbol (empty tick history to start), it takes 90 seconds to execute ! On memory 10 millions ticks is around 600 MB, but they are then compressed somehow to be written in ticks file (database). A database tick being between 6 and 7 bytes, let's say 7 bytes, 10 millions ticks is at worst 70 MB.

Checking my computer's disk, I can see the transfer rate is very low, not even 1 MB/s in average. On a hard disk able to reach 80 MB/s in write mode (checked yesterday with a benchmark software).

I don't know if it has always be like that or if it's something new, but that seems veeeery slow to me, even considering the needed processing.


No answer up to now.

 
Arthur Albano:

So, I tried with WDO$ (mini USD dollar futures  Brazilian BMF Bovespa). Guess what? Different brokers, different database. But no difference from demo and real accounts.

There's also a difference when all ticks are requested (broker M: 11,502,790 vs 13,171,340; broker R: 11,747,636 vs 11,720,021):

Broker R does not store Bid/Ask ticks, but broker M does:

So, my best guess is that some brokers store some information, while others filters data. I am used to operational historian (basically a database server) doing optimization to trash useless information or even condensate them that are not unique. For exemple, if same price arrives with different volumes, it can add the volumes, and use last timestamp. It's called server side compression :P

This is not related to the topic, but it's extremely important remeber to select correct symbol. For instance, an EA placed an Ask value from a different symbol... A real life learning about overusing _Symbol...


 
Arthur Albano:

This is not related to the topic, but it's extremely important remeber to select correct symbol. For instance, an EA placed an Ask value from a different symbol... A real life earning about overusing _Symbol...


How do you know it was an EA ? it was yours ...loool
 
Alain Verleyen:
How do you know it was an EA ? it was yours ...loool

I've coded an EA hedging several symbols... And one of the lines used ask/bid price for a different symbol. I was using demo account, but the above picture is real trading. No, it was not me! :O

 
Alain Verleyen:

Thanks for you interesting comparison. It results in even more questions :-D

I am also working on exchange markets data, in details, for example I already reported an issue about ticks with same timestamp (millisecond precision), being mixed up, with some consequence.

What you said about server side compression is useful, and it should be checked, however all brokers used MT5 server, so it would be surprising they behave differently, but who knows, maybe MQ provided an option to activate it. But anyway, here I have MORE ticks than what is reported by OHLC tick volume, so it's rather a data decompression ;-) Seriously, it's a Forex symbol, without real volume, so I don't think it's relevant in this case.


 

Broker X seems to store only last/volume ticks.


Reason: