6) If tickdata is real data and not
only a simulation, where is it stored then? The lowest timeframe I can get is
1-Minute. However if I observe the strategy tester on this timeframe, I see
movements of the prices. So there must be storage for these ticks?
7) If I want to import tick data into
my MT4 where will this be stored since the lowest timeframe is 1-Minute?
8) My strategy is independent of bars
and indicators. When running the same strategy on exactly the same time horizon
but in different timeframes, I will get different results. How is this
possible? Also, the number of ‘ticks modelled’ varies.
...........
Since my strategy
focusses on ticks (this time, ticks represent a signal not the quality of the
data) and I have limited access to my broker’s data, I would like to use
external data for a strategy tester. If I do, I will have highly different quantities
of ticks between my broker and external data. Do you have an idea how to
simulate the same quantity of ticks with external data so that they are similar
to the quantity of ticks of my broker?
6 & 7) The
tick data is stored in [mt4 data folder]\tester\history\symbol.FXT
file . MT4 does not support direct tick data import. You have to use a
script Birt's csv2fxt or 3rd party tool like TickStory or SQ Tick
Downloader. Before TDSv2, I would use TickStory to download free
compressed Dukascopy tick data, and then generate an updated fxt file
for the year. I would have separate mt4 installation folder every 2
years so I only had to remake the fxt file of the most recent 1-2 years.
TDSv2
has completely streamlined the process. You just wait about 40 seconds
prior to the first backtest of a pair and the FXT file is simulated on
the fly from a centralized database. Much less tedious now to get tick
data....highly recommend TDS if you are serious about backtesting with
tick data in mt4.
8) More accurate to say that you are not using any additional indicators. But there must be some criteria for entry and exit; those criteria can be called indicators. But this depends on what the entry/exit triggers are....you did not elaborate.
Carl Schreiber:
Sounds nice - but
the tickdata files get incredibly big: 1-4 GB lets calculate with 2GB
per file per symbol and per timeframe => 20 symbols and 10 timeframes
=> 400 GB.
Do you know a broker which offers server-space and traffic costs for such an amount of data for all clients?
It sounds nice but it is unfortunately not realistic!!
I
think the dukascopy compressed database is about 25GB-30 for about 10+
years of data across 30+ symbols. Uncompressed (.csv) is about 3-10x
times that. But you only need to decompress locally enough to generate
the csv/fxt file or use in another program. Then delete the csv file.
You technically don't have to hold on to the dukascopy data once you
have generated the fxt file, but I think it is wise to have the
compressed version it stored on a local hard drive.
Programs
like TDSv2 actually eliminate the need for separate FXT file. The
compression is 2x that of dukascopy and there is no more additional
space needed for FXTs....the fxt is simulated on the fly by loading the
compressed tick data directly. But you must wait 40 seconds prior to
starting each tick data backtest of a different symbol or timeframe. I
bet that most programs that use tick data will eventually move towards
this method (hold raw data compressed, and then load it on the fly
without having to generate csv/proprietary format to hold duplicate data
on hard drive).
Once TDSv2 supports exporting data, then
Time
frame is not relevant with tick data. But you would likely use 1min
time frame when backtesting with tick data. Tick level testing is
mainly used by intraday arb/grid traders.
Also, you do not need to download all the tick data at once. You download 2-3 desired symbols and test your strategy.
Alain Verleyen:
Ticks data are ticks data, there is no "timeframe" implied here.
The MT5 platform allows it. If brokers will offer the data is an other matter. Each one can check by itself.
If
Metaquotes have (finally) implemented it, you can be sure it's not "not
realistic" :-D GBytes of data are nothing nowadays (at least in our
countries).
I downloaded 1 year of EURUSD ticks, it took 13 minutes and is using 219.37 Mb.
2016.11.08
18:53:48.691 Tester EURUSD: preliminary downloading of history
ticks completed, 219.37 Mb in 12:55.563 (289.64 Kb/sec)
I would reserve 100GB . With methods like TDSv2, you can use 25-50GB.
Alain Verleyen:.......
They are working on data importing into MT5, so the format will probably become public (?).
Really?
That would be nice addition to mt5. I wonder how you would import the
tick data into mt5....use Tickstory or Birt's csv2fxt script updated for
mt5 when the time arrives? Or is MT5 going to support native tick
data importing?
Allow the use of cookies to log in to the MQL5.com website.
Please enable the necessary setting in your browser, otherwise you will not be able to log in.
You are signed in. Two-factor authentication has been disabled. Set it up again and scan a new QR code, or continue with two-factor authentication disabled.
"Ticks data are ticks data, there is no "timeframe" implied here."
Mt4 you has to have its own fxt-file for each timeframe - mt5 I don't know - is it different?
"I downloaded 1 year of EURUSD ticks, it took 13 minutes and is using 219.37 Mb."
Ok let's start from there.
5 years ~ 1GB & 1 hour for 1 Symbol and one Client.
Guess what (each) broker must be willing to offer for 1000 Clients and 100 Symbols?
But do you know (links, code,..) the data-formate of the tickfiles?
...
5 years ~ 1GB & 1 hour for 1 Symbol and one Client.
Guess what (each) broker must be willing to offer for 1000 Clients and 100 Symbols?
But do you know (links, code,..) the data-formate of the tickfiles?
It's not public as far as I know, but I didn't search.
They are working on data importing into MT5, so the format will probably become public (?).
..............
6) If tickdata is real data and not only a simulation, where is it stored then? The lowest timeframe I can get is 1-Minute. However if I observe the strategy tester on this timeframe, I see movements of the prices. So there must be storage for these ticks?
7) If I want to import tick data into my MT4 where will this be stored since the lowest timeframe is 1-Minute?
8) My strategy is independent of bars and indicators. When running the same strategy on exactly the same time horizon but in different timeframes, I will get different results. How is this possible? Also, the number of ‘ticks modelled’ varies.
...........
Since my strategy focusses on ticks (this time, ticks represent a signal not the quality of the data) and I have limited access to my broker’s data, I would like to use external data for a strategy tester. If I do, I will have highly different quantities of ticks between my broker and external data. Do you have an idea how to simulate the same quantity of ticks with external data so that they are similar to the quantity of ticks of my broker?
6 & 7) The tick data is stored in [mt4 data folder]\tester\history\symbol.FXT file . MT4 does not support direct tick data import. You have to use a script Birt's csv2fxt or 3rd party tool like TickStory or SQ Tick Downloader. Before TDSv2, I would use TickStory to download free compressed Dukascopy tick data, and then generate an updated fxt file for the year. I would have separate mt4 installation folder every 2 years so I only had to remake the fxt file of the most recent 1-2 years.
TDSv2 has completely streamlined the process. You just wait about 40 seconds prior to the first backtest of a pair and the FXT file is simulated on the fly from a centralized database. Much less tedious now to get tick data....highly recommend TDS if you are serious about backtesting with tick data in mt4.
8) More accurate to say that you are not using any additional indicators. But there must be some criteria for entry and exit; those criteria can be called indicators. But this depends on what the entry/exit triggers are....you did not elaborate.
Sounds nice - but the tickdata files get incredibly big: 1-4 GB lets calculate with 2GB per file per symbol and per timeframe => 20 symbols and 10 timeframes => 400 GB.
Do you know a broker which offers server-space and traffic costs for such an amount of data for all clients?
It sounds nice but it is unfortunately not realistic!!
I think the dukascopy compressed database is about 25GB-30 for about 10+ years of data across 30+ symbols. Uncompressed (.csv) is about 3-10x times that. But you only need to decompress locally enough to generate the csv/fxt file or use in another program. Then delete the csv file. You technically don't have to hold on to the dukascopy data once you have generated the fxt file, but I think it is wise to have the compressed version it stored on a local hard drive.
Programs like TDSv2 actually eliminate the need for separate FXT file. The compression is 2x that of dukascopy and there is no more additional space needed for FXTs....the fxt is simulated on the fly by loading the compressed tick data directly. But you must wait 40 seconds prior to starting each tick data backtest of a different symbol or timeframe. I bet that most programs that use tick data will eventually move towards this method (hold raw data compressed, and then load it on the fly without having to generate csv/proprietary format to hold duplicate data on hard drive).
Once TDSv2 supports exporting data, then
Time frame is not relevant with tick data. But you would likely use 1min time frame when backtesting with tick data. Tick level testing is mainly used by intraday arb/grid traders.
Also, you do not need to download all the tick data at once. You download 2-3 desired symbols and test your strategy.
2016.11.08 18:53:48.691 Tester EURUSD: preliminary downloading of history ticks completed, 219.37 Mb in 12:55.563 (289.64 Kb/sec)
I would reserve 100GB . With methods like TDSv2, you can use 25-50GB.
They are working on data importing into MT5, so the format will probably become public (?).
Really? That would be nice addition to mt5. I wonder how you would import the tick data into mt5....use Tickstory or Birt's csv2fxt script updated for mt5 when the time arrives? Or is MT5 going to support native tick data importing?