Can variable Spread back-testing/optimization in MT4 be achieved without tick-data?

 

I am trying to implement variable spread back-testing/optimization, and after doing some research so far I don't think this is possible. It seems to me that back-testing/optimizing using variable spreads will always translate to using tick data.

Unfortunately the TDS option won't work for me since I am launching several MT4 instances in batch mode. But I came across a very elegant way of using custom FXT files by adding a piece of code to the EA that rewrites the MT4-generated FXT with the custom one before the actual back-test begins (http://77z3-fx.com/2012/10/03/mt4-backtesting-how-to-use-your-own-fxt-files-without-using-the-downloaded-data/).

So I decided to test this technique:

* I recorded some 50 days (Feb to April 2013) of AUDNZD tick-data using Geektrader's script

* I then created custom fixed spread FXT and HST files (with spread 3 pips) for AUDNZD M5 between 01/03/2013 and 31/03/2013, using Birt's CSV2FXT script. My MT4 was online as required for this, but after that all back-tests were done offline.

* I modified the Moving Average EA adding the FXT-writing code.

Unfortunately my tests ran into some strange problems:

1) when back-testing the March period using the custom FXT and 1.HST/5.HST files, the test was aborted with error "unmatched Data errors". The custom FXT was not copied over the generated one, and ST flagged invalid file for AUDNZD5 test.

2) I then back-tested using the original MT4 1.HST/5.HST files. The test was completed, but trades for day 5 and half of day 6 were omitted. This is a problem because back-testing the normal Moving Average EA with the MT4's M1 data does show triggered trades during these times.

3) finally I back-tested using only the custom FXT and 5.HST files. The test was also completed (with modelling quality 25% as expected), but this time some trades where triggered "ahead of time" (the report lists trades on day 7 that happened "before" day 6)

Given that back-test 1) did not complete, but back-test 2) did complete, I am suspecting that the generated HST files are "not in agreement" regarding, ticks, volume, etc. But this is hard to believe since Birt's CSV2FX script is well established...

The original csv file is too big to be attached, but I have attached in a zip file the generated HST files, the modified Moving Average EA, as well as reports for back-test 1) (ReportA) and back-test 2) (ReportB). I will attach the custom FXT file in the next post.

Why is this not working?! Has anyone tried this before?! Did I miss anything?! :-)

Files:
 

Here is the custom FXT file I used. If someone with TDS could check the integrity of these custom FXT and HST files that would be great! :-)

Thanks!

Files:
audnzd5_0.fxt.zip  4070 kb
 
pipnotic:

Here is the custom FXT file I used. If someone with TDS could check the integrity of these custom FXT and HST files that would be great! :-)

Thanks!

Do fxt files produced with one Broker's terminal work on another Broker's terminal ?
 
RaptorUK:
Do fxt files produced with one Broker's terminal work on another Broker's terminal ?


Sorry, RaptorUK, could I ask you to elaborate on that?

In a normal back-testing the FXT file is created from the HST data available on the selected broker's history folder. Are you asking if the FXT created from the history data of one broker will work with the HST files of another broker?

 
pipnotic:


Sorry, RaptorUK, could I ask you to elaborate on that?

In a normal back-testing the FXT file is created from the HST data available on the selected broker's history folder. Are you asking if the FXT created from the history data of one broker will work with the HST files of another broker?

Not precisely, I'm asking if you create a fxt file on the terminal of BrokerA and then copy it to the terminal of BrokerB will it work ?
 
RaptorUK:
Not precisely, I'm asking if you create a fxt file on the terminal of BrokerA and then copy it to the terminal of BrokerB will it work ?


Hi, RaptorUK

Still struggling to understand your question... Regarding creating FXT files, we have two scenarios here:

1) FXT files created automatically by MT4 when a back-test/optimization is started. MT4 creates it based on the HST files it downloads from the broker.

2) FXT files created from downloaded tick data using the CSV2FXT script. Here the FXT is created while the terminal is online (therefore connected to a broker), but once the FXT and HST files were created the tests are done with the terminal offline.

If you are referring to scenario 1), I could try to generate an FXT file using the history from BrokerA, and then run a back-test using this "custom" BrokerA FXT with the HST files of BrokerB. I just cannot really understand the reason for that...?!

If you are referring to scenario 2), are you thinking there might be a problem with the running of CSV2FXT with BrokerA?

Sorry that I am still not getting your point...

 
pipnotic:


Hi, RaptorUK

Still struggling to understand your question... Regarding creating FXT files, we have two scenarios here:

You posted a fxt file in the 2nd post in this thread . . . will it work with any Broker's terminal ? have you tested it with other Brokers ?
 

The FXT file I posted was generated with the CSV2FXT script, and therefore it is based on tick-data and not HST data from any broker.

In theory it should work on any MT4 terminal when together with the HST files generated by the same script, especially considering that the terminal is offline for the tests anyway...

 
pipnotic:

The FXT file I posted was generated with the CSV2FXT script, and therefore it is based on tick-data and not HST data from any broker.

In theory it should work on any MT4 terminal when together with the HST files generated by the same script, especially considering that the terminal is offline for the tests anyway...

Yes I know . . . shall I tell you what Broker's terminal you used to run the CSV2FXT file in ? AxiTrader-US03-Demo

The last time I used tick data with MT4 was with a evaluation TDS and I was having issues getting it to run, I never got to the bottom of the issue but I suspect it was because I had prepared my fxt file and hst files on a terminal from BrokerA and was trying to use the fxt and hst files on BrokerB . . . I know the hst files will work without issue, they don't contain any reference to the Broker, but the fxt files does, I wonder why that is.

Reason: