How to choose Timeframe in the Tester? An experiment.

New comment
 

Although I have been reading all information I can get my hands on regarding back-testing in MT4 (including all Tester related articles in the articles section), I still can't understand timeframes in the MT4 Tester...


To better understand the significance of choosing timeframes in Tester I have done a little experiment:

1. I manually downloaded M1 Alpari data via just scrolling back in the M1 chart (arbitrarily choosing EURCHF for this experiment).
2. To get other time-frames I converted the M1 data using the script that comes with MT4 (by opening the M1 data offline).
3. I used the simplest possible expert to write tick data during testing to file (the expert simply writes date-time and value of bid at each run of start() to a CSV file):


extern string name = "test.csv";
int handle;
int init()
  {
   handle = FileOpen( name, FILE_READ|FILE_WRITE|FILE_CSV, ',');
   return(0);
  }
int deinit()
  {
   return(0);
  }
int start()
  {
   FileWrite( handle, TimeToStr(TimeCurrent(), TIME_DATE|TIME_SECONDS), DoubleToStr(Bid,5) );
   return(0);
  }

4. I set the Tester to run the expert on one week (arbitrarily choosing 26-10-2009 to 2-11-2009).
5. While testing I made sure internet is disconnected so as all runs will have same market properties.
6. I run this test for each timeframe on "Every Tick" model and got output files for all timeframes.


Results:

First, I opened all FXT files that where produced during Testing in offline mode – they look different when seen as Candlesticks or Bar charts but look almost identical as Line charts (the differences are VERY small).

But I don't really care what they look like, what I care about is when start() runs and what Bid price does it "see"… Cause that's what any algorithm I run is going to be working on.


Comparing the output files:

I run a diff on the files using Beyond Compare. Here is a summary of results:

  • Left M1, Right M5: 264452 same row(s), 2374 important left orphan row(s), 1765 important right orphan row(s)
  • Left M1, Right M15: 265236 same row(s), 1590 important left orphan row(s), 873 important right orphan row(s)
  • Left M1, Right M30: 265435 same row(s), 1391 important left orphan row(s), 655 important right orphan row(s)
  • Left M1, Right H1: 265531 same row(s), 1295 important left orphan row(s), 544 important right orphan row(s)
  • Left M1, Right H4: 265602 same row(s), 1224 important left orphan row(s), 458 important right orphan row(s)
  • Left M1, Right D1: 265624 same row(s), 1202 important left orphan row(s), 434 important right orphan row(s)
  • Left M5, Right M15: 265223 same row(s), 994 important left orphan row(s), 886 important right orphan row(s)
  • Left M5, Right M30: 264970 same row(s), 1247 important left orphan row(s), 1120 important right orphan row(s)

And so on…


Looking at the diffs themselves it is easy to see that there are only 2 types of differences:

1. Lower time frames have extra ticks spread out in the data (the lower the time frame the more there are). All extra ticks seem to have the same Bid value as the adjacent tick (the one before or after them).

2. Some ticks have a +/- 1 second difference in their time (these account for the majority of the differences).


Looking at the diff data it can be seen that at most ~1.5% of ticks are different between 2 timeframes and usually less than ~1%. The differences themselves (although I did not measure them) are so small that I can safely say they can be ignored.


I am not sure why there are more ticks in the smaller time frames or why there are some ticks with +/- 1 second difference in time (maybe due to the conversion process between M1 and the other time frames?), but for all practical purposes the tick data while using different time-frames is IDENTICAL. I easily confirmed this by running any of my EA's on history data with different time frames - all had practically the same results (give or take a few cents in profit, or a second or two in open or close times).

So I have some questions for the forum GURU's or for MetaQuotes, the questions that led me to do this little experiment:

1. WHAT is the difference between the timeframe settings in the Tester? I don’t care that the chart looks different. My EA does not "look" at the chart, it's just reading tick data and for all purposes I have proved that tick data is almost identical no matter what time frame u use. So why choose any timeframe over another?

2. People seem to be really bothered by "Modeling Quality" while Testing. I don't get it - if tick data while Testing on M1 (which has modeling quality of 25% only) is almost identical to the tick data while testing on ANY other timeframe (which give 90% modeling quality), what does "Modeling Quality" mean? Why should I care about it?

 
Anybody?
 

Hi Gordon


Interesting work. My humble thoughts:

1. The timeframe setting on the tester selects the timeframe of the chart that the EA is applied to, i.e. to simulate what would happen if you applied the EA to a chart of that timeframe. If your EA is written to be independent of the chart timeframe, e.g. iHigh(Symbol(), M15, 0) instead of High[0], then it doesn't matter what chart you apply it to and the tester timeframe setting doesn't matter.

2. To understand Modelling Quality refer to https://www.mql5.com/en/articles/1486 and https://www.mql5.com/en/articles/1513. Every time I see a response to questions about modelling quality, the experienced forum members say don't worry about it. The formula given in the article shows that the modelling quality on M1 can't be better than 25%.


Cheers

Jellybean





 
Well I never even thought of iHigh(), iLow(), etc. My EA's don't use any of those functions at all. But I was sure all timeframes are accessible via iHigh(), iLow(), etc. regardless of the timeframe in the chart the EA is attached to?
 
gordon:
Well I never even thought of iHigh(), iLow(), etc. My EA's don't use any of those functions at all. But I was sure all timeframes are accessible via iHigh(), iLow(), etc. regardless of the timeframe in the chart the EA is attached to?

Hi Gordon


Yes, the functions iHigh(), iLow, iTime, etc. access the specified the values for the specified timeframe, regardless of the chart they are attached to. If you write you EA with High[], Low[], Time[], etc. the values depend on the timeframe of the chart they are attached to.


You could demonstrate it in the same way you've tested the ticks in the tester. If you write an EA to write Time[0] and iTime( Symbol(), D1, 0 ), for example, you should see Time[0] dependent on the tester timeframe setting, whereas iTime will be independent of it.


Cheers

Tony

 

Thanks Jellybean. That's definitely the answer to my first question. I am familiar with iLow(), iHigh(), etc. functions, although I rarely if ever use them, but I never got to use High[], Low[], etc. so was not aware of how they behaved. Great! At least that clears up some mystery.


Well, I guess since I don't use High[], Low[], etc. I am just going to use whatever timeframe and ignore the "Modeling Quality"... Unless somebody can show me any real difference between them (besides the behavior of High[]...). It seems to me that the whole "using higher timeframes is better" is just a myth, or a misunderstanding (or maybe somebody did a bad translation from Russian :).

 

Hi Gordon

My comments don't apply just to the standard High[], etc.

All of the indicator funcitons, e.g. iMACD() allow the timeframe to be specified, including the option of "NULL". If you spedify "NULL" for the timeframe the value returned will depend on the timeframe fo the chart it is attached to. However, specifing a specific timeframe will return the value for that timeframe regardless of the chart timeframe.

Cheers

Jellybean

 
Good to know. Thanks.
New comment
Reason: