Is the 2GB limit for FXT files still around? - page 5
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Thanks to all the contributors for a meaningful article.
I have also tested and can confirm the 4GB limit on fxt files running on Win Xp 32 bit with MT4 build 509.
Since the simulation uses the M1 data if available, my solution was to adjust the volume on the M1 history file.
I believe the tester simulates price by generating ticks to represent the Open, High, Low and Close of each M1 bar and then generate extra ticks in between until the number of ticks matches the volume of the M1 bar.
My solution was to tune down the volume and you have fewer ticks and this brings the required test data span within the 4GB limit.
Since the ticks are anyway simulated and does not represent real data except for the OHLC, is there any sense in having volume over, say 25 on a M1 bar??
A smaller number may be enough to accurately enough reflect price on a single bar, obviously depending on the type of strategy you are testing.
I comfortably tested 4,5 years worth of M1 data with an fxt file size of 1,5GB - plenty of room to grow!!
An added advantage is that the test runs much faster since the execution speed is determined by the number of ticks generated. Faster testing makes for faster optimisation!!
To maintain consistency and the 90% modeling quality figure (if that is important to you) you should also adjust the volume of the higher timeframe bars such that you have volume integrity between the time frame used for testing and the M1 bars.
Happy testing!!
Hi fxapie,
I'm so glad you are asking the same questions as me regarding how/why MT4 uses the tick data in relation to the M1 bars.
You'll want to read this topic: https://forum.mql4.com/56026
I'm yet to find any answers, which is a shame as if they can be found, it potentially opens up testing across ranges of many years using 'tick-efficient' M1 data (for those strategies that use it).
Trev.
Hi fxapie,
I'm so glad you are asking the same questions as me regarding how/why MT4 uses the tick data in relation to the M1 bars.
You'll want to read this topic: https://forum.mql4.com/56026
I'm yet to find any answers, which is a shame as if they can be found, it potentially opens up testing across ranges of many years using 'tick-efficient' M1 data (for those strategies that use it).
Trev.
I think the type of strategy and the trading timeframe you are using may have an influence on your decision regarding history data. In my case I tested a M5 strategy and found NO difference in the results when using original data and volume scaled down data. Same nr of trades, same profit, drawdown, etc. This was not using raw tick data (did try that before as well)
Bear in mind that tick data still must honour the OHLC of the M1 bars. Does it really matter how price moved between those 4 levels???
Of course it will be important if you are scalping on M1 but on higher timeframes which are built on M1 as source I am not convinced that tick data or original volume simulated ticks has any advantage.
But as you rightly said, each one must make their own choices and live with the results.
Hi fxapie,
I'm so glad you are asking the same questions as me regarding how/why MT4 uses the tick data in relation to the M1 bars.
You'll want to read this topic: https://forum.mql4.com/56026
I'm yet to find any answers, which is a shame as if they can be found, it potentially opens up testing across ranges of many years using 'tick-efficient' M1 data (for those strategies that use it).
Trev.
Hi Trev
Thank you for the pointer. I still believe that if you have M1 data for the period that you want to test and you are working on a higher timeframe (M5 and up) then reducing volume can work and has advantages. The data generated must still reproduce the OHLC of the M1 bars which build the higher timeframe bars.
Testing on M1 with reduced volume will certainly affect the outcome.
Will test and verify when have a gap.
Nico
If you want to use simulated ticks that are worse that the default simulated ticks then you can and you can take the risk . . . I prefer to use real tick data where I can, if not I prefer to use the best simulated ticks I can get access to.
Hi Raptor. I love your style :)
Since disparate brokers can potentially produce the exact same M1 price action (ohlc) on a given instrument, and yet build their M1 bar's tick data differently to one another, what importance should be placed on the actual number of ticks? Alpari ticks FTSE every 0.5pt movement, whilst FXCM every 1pt. That is to say, the tick data we get from brokers is merely representative of their particular setup. So which one is 'real'? Neither and both.
Furthermore, without knowing exactly what MT4 does with these ticks in terms of interpolation, then we can't assess its real importance.
Upon testing (as per the other thread), my best guess is that each M1 bar needs circa 1 tick per broker increment/movement in order to be 'filled to its extremities', as it were. I may be wrong and I'd like to be better informed but I'm not sure anyone here really knows. If it is true however, then replacing it shouldn't make any difference to the testing outcome (of a strategy that doesn't need more granularity than M1 ohlc), and it would be great to write a script that assigns the correct minimum tick volume per M1 candle.
Remember that the reason I'm investigating these things is that I was given FTSE price data from a prop house that had market depth information in the volume column rather than tick volume (making it useless to MT4 and causing .fxt issues). I didn't want to just junk the dataset because of this. Firstly, it goes further back in time than my next best dataset. Secondly, I wanted to prove that my EA works with their price feed. So naturally, I wanted to understand the consequences/risk in MT4 testing terms of changing/replacing that volume data.
All ideas gratefully received.
Hi Nico. Thanks. Yes, it's probably down to scaling appropriate to the bar's time-frame and height and I imagine accuracy is more important the smaller the timeframe.
Edit - now superseded by the comments below.
Hi Raptor. I love your style :)
Since disparate brokers can potentially produce the exact same M1 price action (ohlc) on a given instrument, and yet build their M1 bar's tick data differently to one another, what importance should be placed on the actual number of ticks? Alpari ticks FTSE every 0.5pt movement, whilst FXCM every 1pt. That is to say, the tick data we get from brokers is merely representative of their particular setup. So which one is 'real'? Neither and both.
Furthermore, without knowing exactly what MT4 does with these ticks in terms of interpolation, then we can't assess its real importance.
MetaQuotes tells you how they calculate the ticks: https://www.mql5.com/en/articles/75
I take your point about different Brokers doing different things, my point is that I don't want to make the situation even worse than it currently is . . . if I can help it. I have compiled a little data already about ticks coming from Brokers, you might find this interesting . . . it's tick count for each H1 bar on the Y axis and time along the X axis
Well I'll be. Thanks Raptor, the description in the article is exactly what I have been looking for. It answers my question perfectly and improves my understanding of the whole simulation ideal (and the apparent problems with it). Is this article available on this site or is it only resident on MQL5? It could be the reason I'd not seen it previously.
Now I can potentially* have a script written that analyses a 1min dataset and generates/inputs the correct minimum amount of ticks for each candle, outputs to a .csv, then imported so that MT4 can apply its simulation properly (even if it's not as good as MT5). Like you said earlier, better to have the tick count that came with the dataset where possible but in the absence of such (like in the situation I'm faced with this external dataset), it should be the next best thing and good enough in any case. In some instances, it could improve the situation.
*I say potentially because not being techy, I don't know how difficult this is to do. What does seem possible to do quite easily is work out what the maximum number of ticks any candle might need based on the number of support points. Which is pretty much what I established (without knowing it), in my experimentation in the other thread (although I only have an approximation).
As for the H1 tick count graph, that's really interesting. 10,000 ticks in one hour on FXCM though? I have just looked through that time period in FXCM in my history centre. I exported the 1min data from 9am to 11am on 22nd April 2013 and for the whole two hours there are only 577 ticks? FXCM have told me in the past that their data-feed is in the singular so I assume there's no account-type discrepancies (between CFD and spread bet accounts etc). I must not be understanding something quite right.
As for the H1 tick count graph, that's really interesting. 10,000 ticks in one hour on FXCM though? I have just looked through that time period in FXCM in my history centre. I exported the 1min data from 9am to 11am on 22nd April 2013 and for the whole two hours there are only 577 ticks? FXCM have told me in the past that their data-feed is in the singular so I assume there's no account-type discrepancies (between CFD and spread bet accounts etc). I must not be understanding something quite right.