debugging MQL5 EA using artificial tick data - page 2

 
Icham Aidibe:

Err ... 

CSV format is a well known format. But MT5 seems to need the tick flag (bid, ask or bid/ask) to know which kind of ticks it'll generate. 

Here's the ticks as MT5 need it : 

Here's tickstory with data formating matching MT5's one. No tick flags. No last prices (=0 used).

Then, add the header as MT5 read it : 

Import the csv in MT5 : 

As expected everything's correctly filled but the flag, volume and last.

Then run any EA on the custom symbol and ..... BEEP. Nothing happens. 

Retry with the "based on real tick mode" and ..... BEEP. Nothing happens.

So ... 

  • Brokers delivers one year maximum of good tick data quality : definitively not enough to be forwarded 
  • Importing data on 5 years or even 10 from another source (dukascopy is a reference), with 2 main disadvantages : 
    • First, it doesn't work actually (unless I made a error in the procedure, please correct me) 
    • Second, even if it worked, it's not the brokers data, none won't win in accuracy
  • The only interest I find there is to collect good quality ticks from a year to another and use it in custom symbol, but brokers don't keep these huge files on their server - and so I won't in my laptop. 

That "based on real tick" mode is ... a try, a ideal, a utopia. markets aren't frozen.

qED.

So @shloss is right, it's not a problem of importing data, you did it successfully too. It's a problem of being able to use them to backtest, don't forget it's very new feature, still in development. It will certainly improve on next builds.
 
Alain Verleyen:
So @shloss is right, it's not a problem of importing data, you did it successfully too. It's a problem of being able to use them to backtest, don't forget it's very new feature, still in development. It will certainly improve on next builds.

It is about datas ... there's no "tick flags" on tickstory and others datas for example, MT5 needs it to know which kind of tick to generate. We may ask the coders team to be sure, but it's pretty obvious. 

Look I'll import from tickstory, and past in the flag column the ticks flag from MT5. Quasi-100% luck to work. Let me test. Just to be sure.

 

1. Export ticks from MT5 brokers in a CSV

2. Replace FLAGS values by a null value like in any others datas provided (tickstory)

3. Reimport the set with no flags like once again, you would have downloaded it from another source

4. MT5 can't use it.

[EDIT]

I tried to import from tick story with the "Bid/Ask" flag as a default hoping that if a value is the same, it simply won't change. Using real tick mode, looks like ticks aren't saved for the custom symbol.

I tried to import from MT5 native tick history. Using real tick mode, looks like ticks aren't saved for the custom symbol.  

 
Icham Aidibe:

1. Export ticks from MT5 brokers in a CSV

2. Replace FLAGS values by a null value like in any others datas provided (tickstory)

3. Reimport the set with no flags like once again, you would have downloaded it from another source

4. MT5 can't use it.

[EDIT]

I tried to import from tick story with the "Bid/Ask" flag as a default hoping that if a value is the same, it simply won't change. Using real tick mode, looks like ticks aren't saved for the custom symbol.

I tried to import from MT5 native tick history. Using real tick mode, looks like ticks aren't saved for the custom symbol.  

And what timeframe are you using ? Even using real ticks, you need bars data. So you have to import/create them.

Backtesting using a custom symbol and real ticks :


They are still some problems as actually I used MT5 data from EURUSD (export) to create a custom symbol (import ticks and bars). Still it gives some "real ticks absent" or "ticks prices not matched 1 minute bars". But it works.

 
Alain Verleyen:

And what timeframe are you using ? Even using real ticks, you need bars data. So you have to import/create them.

Backtesting using a custom symbol and real ticks :


They are still some problems as actually I used MT5 data from EURUSD (export) to create a custom symbol (import ticks and bars). Still it gives some "real ticks absent" or "ticks prices not matched 1 minute bars". But it works.


Okayyyyy ticks data + minute bar data ! gonna try it ! Thanks ! 

[EDIT] : 

It indeed works when adding also the data bars. And using tick story, it seems as their no error in the debug tab, that it's a 100% quality, using Bid/Ask for each ticks as a tick flag.

The spread is the minute bar one. 

Trying with old ticks data from 2015, all datas are from TickStory, the spread is fixed I don't know how to otherwise with tickstory. Note that the tickvol isn't really the tickvolume, but it shouldn't matter .... 



 The tick file from 2015, fresh out from TickStory : 

It imports well... 

But here's the result in the tester : 

 2] 

[EDIT 2] : the spread is fix, the bid & ask are at each tick different :-/////// it definitively require a update !

 
Icham Aidibe:

...

[EDIT 2] : the spread is fix, the bid & ask are at each tick different :-/////// it definitively require a update !

What spread fixed ? It's custom data, you decide of the spread !

You can even change data manually in this window.

 

I can't download minute bar data from 2015 with my broker. So I used tickstory bar data. And with tickstory, I've fixed the spread to 10. But even 0 doesn't work and produce the same result.

It can't be synced, you fix a spread to 10, the terminal read a bid+25pts's one. 

 
Alain Verleyen #:

What spread fixed ? It's custom data, you decide of the spread !

You can even change data manually in this window.

About changing data manually:
If you want to change several fields you have to edit that field for every row manually from here.
But say you want a fixed spread for all entries you have to export the file and write some script that changes that entry for each row and then import the modified file. 
 
Gunnar Forsgren #:
About changing data manually:
If you want to change several fields you have to do every one manually from here.
But say you want a fixed spread for all entries you have to export the file and write some script that changes that entry for each row and then import the modified file. 
Excel can help to modify at bulk.

Exporting to the right CSV file format might be a bit tricky though. LibreOffice is better for that.


 
Dominik Christian Egert #:
Excel can help to modify at bulk.

Exporting to the right CSV file format might be a bit tricky though. LibreOffice is better for that.


Yes I just figure I use some scripting language I am familiar with. That symbols list above by Alain explains the format well.
Reason: