Backtesting it's a joke?

New comment
 
Really I don't understand, when I was absolute beginner I use to get different results testing my ea with same parameters, then I understand that the chart have to be error free, ok, now the chart is error free so why the results of the evening are exactly the opposite of the one of the afternoon? Same parameters, same period, every tick for both and the results are completely different! Please help me to understand before I get mad. Thank you!
 

Hi Mario, to answer such questions youself you need to eat many many breads and read books and experiment. backtesting is a joke if you dont understand the details.



Depending on idea of your strategy answer may be different.


-Imagine your EA is trading the equity curve, then results will depend on previous results and if you start your EA 100 bars later you may begin with DD instead of PRofit (on morning) and your EA may decide to stop trading for a while, this way trading curve may look very different. But this is more advanced topic.

-Your broker may have corrected 1 peak and you had this peak as winner which was biggest earner

- If you really want do backtest, then pick another tool which is pure backtest oriented. I use tradestation and wealth-lab. MT4 is great tool for automating but unfortunetly backtesting skills are missing. I assume those capabilities are cut with intention. I hope MT5 will be better


What kind of strategy does your EA follow?

 

In order to achieve consistent results with the strategy tester, you need to lock down all the variables.

Basically, you need to ensure that your history data and parameters do not change.

One example of how you may innocently skew a test would be to perform your 1st test, then have the tester platform online, allowing the spread to change. Then the changed spread may be used for your 2nd test. So one of the important steps is to ensure that your tester platform never connects to a broker.

If you do a search, you'll find some good step by step guides.

It takes effort and patience to have any trust in strategy tester results, over and above just using it for EA functionality testing.


CB

 

As well as spread, there are a number of other things from MarketInfo which could change and cause the results to be completely different.


But rather than showing backtest to be a joke, it actually indicates that the EA concerned is very sensitive to minor variations in input and therefore not a good candidate for running with real money. On the other hand, certain types of EAs actually thrive on and take advantage of these variations, but that probably doesn't apply in your case. :)

 

A quick fix to this problem is to tick "Enable Proxy Server" under Options>Server, in the Proxy field enter a obviously none-existent server (like 1.1.1.1) and restart MetaTrader. From now on it won't connect anymore to the server so as Market Properties (spread, etc.) won't change. Test whatever u want and when u r finished untick "Enable Proxy Server" to reconnect.

Another thing is to manually delete all folders under \tester\ folder (while MT4 is closed) once in a while. Over time there is a tendency of some files in these folders to get corrupted and foul up tests. Good luck!

 
Thanks to all, obviusly I performed several tests without change any parameter but I didn't consider the event that the spread could change, in fact my ea is a scalper, so the goal is to get minimun but frequent profit and minimum variations on the market could be fatal (now I understood thanks to you), as I said I ran the test on EURUSD at 16:00 from 01/2009 till oct 20 2009 and the account was rising from 5000 eur to 17000! I tested again at 21:00 and the account falls to 0 2 month simulation later. That's why I felt frustrated. After sent my first post I ran the test again with GBPUSD.... surprise! It's was working as in the afternoon, trying again with EURUSD and nothing to do, still loosing. Surely it depend by broker spread. I'll try to pick your precious advice. Thanks again
 

> Surely it depend by broker spread

Thats it - if you press the Symbol Properties button after your backtest, you will see the Spread used during that test

It varies with the live spread at that moment, not the spread in play during the hours/days of the backtest period

When the market is closed, a default spread is used that is relatively low - dont know where that comes from?

Maybe one of MTs .sel files...?

I suggest your EA does a check on the spread before entering a trade

Also look at which trading hours are best for your strategy

FWIW

-BB-

 

Recently, I realized that my strategy tester has to stay online before I could do the backtesting... I use Alpari & FXDD.

And, if i stay online, i am risking my downloaded historical data...

Any walk around?

 

I am about to test the method prescribed by birt on his EA review blog which uses a method to bypass metatraders creation of psudo tick data by creating a replacement backtester history file of real tick data from a source of known good quality source.

I'll let you know how well this works.

 

SDC, if you already know the MT4 platform doesn't ensure every tick sent by the broker will be received, logged, and treated as actionable info by your specific instance of the platform then what good is it to backtest with "real" tick data versus "fake" tick data?

Neither case is doing any better of a job of re-creating the market environment your EA would have experienced were you using it in real-time back in time.

 

I thought it would because the real tick data comes direct from that brokers own price quote server so it should be the exact same as the tick data their price feed server issues, and secondly the way I understand it, mt4 creates psudo ticks, estimated from the mt4 1 minute chart history and creates a strategy tester history file of those estimated ticks, where as the custom made fxt file from the downloaded tick data is created from real ticks and that file is used in strategy tester instead of mt4's usual one made of estimated ticks, that is how i understand it I hope someone will correct me if I am wrong.

Haveing said all that my main reason for doing it isnt because i doubt the reliability of the regular strategy tester file so much, it is mainly because my platform is lacking the neccessary history data i need for backtesting and it seemed to be the best way to fill it all in

123
New comment
Reason: