Since I had never used the blog offered by MetaQuotes and since it's a recurring difficulty for anyone testing an EA (self-made or bought) I thought it was an opportunity to share my tips about.
But first, know that English isn't my main language, so be indulgent.
What is curve-fitting ?
Over-optimization (curve fitting) is a process of adjustment the indicators' parameters to the concrete data. Computer technology can be easily used to over-optimize a trading system and produce something that looks good. An optimizer tests thousands of parameters value and can find the exact numbers making the optimizing strategy very "profitable". The problem is that "Profitable" strategy is profitable only applied on this data. However, it is highly possible the system to fall apart in real world. -someone on the forum
Concretly ?
- I select the MACDsample
- I optimize it on the first week of year 2017
- I select the best result
- I run a test with the best set - it's perfect
- So I make a deposit & set the EA
- I wait one week : it's ok !
- I wait 2 weeks : Damn, still ok, I'm in !
- 3, 4, 5 weeks etc ... The account took 8 months to burn, but it did burn !
This is curve-fitting : it's a very common experience.
Tomorrow nevers dies
Stress test : stability over time (1 random week, each semester, since 2015)
It means nothing 'til now. Let's deepen.
Stress test : one year (2017) on 3 differents brokers
Broker 1 without commission, broker 2 with a commission of 16UI, broker 3 with a commission of 20UI (UI = International units (to simplify!))
It's now reasonable to rule out any curve fitting, even if only a year has been tested.
Stress test : long term (6 years) & differents brokers
Let's check it on long term : brokers' I'm using provide 99% quality datas since year 2012 (6 years), so let's do with what we have :
Broker 1 without commission (from 10/2012) & Broker 2 with commissions (from 01/2012); brokers 3 (bad quality in in 2012)
Here, we can start talking of a "winning strategy".
Ultimate stress test : platform
So, if it's a winning strategy, by applying the exact same algo, with the exact same params I should obtain a same/very-alike result anywhere, whatever the language used to code it, would you tell me ? And you're true !
Let's complicate the things, checking that fact by running the exactly same translated code on another platform with which it's possible to accomplish the exact same things (MT4), with the exact same parameters, on data matching broker 1's specifications with a similar spread of 10pts (broker target spread floating between 5 - 7, from 6am to 10pm) :
Okay ! Alike broker 2's performance (with commissions) - even if the data sample is commission-free, spread being overestimated : results are in accordance.
What about the forward test ?
The forward test (accessible thru the tester), or the forward back test and all the variants are of course efficient. It's the classic way to avoid it, but keep in mind, that a forward-test (classical one !) can also suffer of overfitting !
And voilà !
Feel free to contribute and to enrich by your own methods what finally appears to be useful for everyone having/planning to run an EA.
