Why backtests are useless, EAs are flawed and their parameters are bad [DISCUSS] - page 2
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Walk Forward Analysis - the only logical successor to backtesting [DISCUSS] [PART 1]
BECAUSE OF A 10.000 CHARACTER LIMIT PER POST; THE SECOND (AND INTERESTING) PART CAN BE FOUND HERE: https://www.mql5.com/en/forum
Hello,
I'm Darwin and today I want to explain how a walk-forward-analysis works and why it is the only logical way to analyse EAs.
I know that the title is a bit provocative, but it's often easier to start a discussion with a controversy.
Also, I know that the best EA-traders can use normal backtesting and still be profitable - but most can not. And even if you can, a WFA is still better.
I WILL RELEASE A WALK-FORWARD-ANALYZER FOR FREE WITHIN THE NEXT FEW DAYS THAT DOES ALL OF THIS 100% AUTOMATED; UNTIL THEN, USE THE TIME TO MAKE YOURSELF FAMILIAR WITH THE CONCEPTS.
Nevertheless, none of this article is needed to use the walk-forward-analyzer tool, it will be as easy to use as the metatrader4-backtester.
After reading, you have 2 choices:
Agree with my arguments and once and for all get rid of the flawed backtesting-approach and use WFA in the future.
Disagree with my arguments, but then please try to argue with me.. Do yourself a favour, don't just stick to backtests because you "know" them or something like that.
Also, please read my first article: "Why backtests are worthless, fixed-logic-EAs are flawed and your parameters are bad [DISCUSS!]" (here on the forum)
Initial Situation
The 3 parts of each trading-system.
1. The system's logic
The most obvious part! And for a lot beginners it's the only part they know, which is dangerous.
This might be a manual trading system or an expert advisor or any other form of fixed trading-logic / trading-system / trading-strategy (btw: all 3 terms name the same thing in this article)
But you all know that every strategy has some kind of variables/parameters (like the periods of moving averages or SL levels etc), that are NOT FIXED(!) but can vary, which brings us to the second part.
(If you just set them to a fixed value because "this should work".. well, it wont, at least not in the long term)
2. The system's parameter-ranges
The ranges of the parameters are an ESSENTIAL part of every trading-system, as they determine the exact behaviour of it (but the trading-logic always stays the same).
So, a moving-average-period might range from 5-15 to capture short-term price movements.
It is not 6 and not 11, it is 5-15, as the markets change, we can't choose a concrete value, ALWAYS a range!
3. The market, the amount of data, the desired characteristics
Every strategy trades on a market, so we want to determine on which. (eg EURUSD / H4)
But thats not enough, we also have to determine how many past-price-data we want to use to evaluate our possible parameter-choices.
Because, as I said, a system always has parameter-ranges, but for live trading we have to choose concrete values!
And we do this by evaluating all parameter-possibilities on the last X years of price-data, and after evaluation,
we end up with a huge list of possible and "independent" trading-systems (each of them with different parameters, but the same main-logic).
And each has it's own characteristics like "profit" or "profit factor" or "relative drawdown".
So, we also have to determine how to pick the "best" parameters.
But it's not as simple as saying "I want much profit", because the characteristics often don't hold in the future!
Instead we want to choose in a way that gives us a high probability of picking parameters that will succeed during live trading.
Simple, isn't it?
An illustrating example
The system's logic:
Let's suppose a very basic trading-system: "If the price moved more than X pips in the last Y days, a course correction will happen"
(just thin air!).
The parameters would be X and Y in this case.
The system's parameter-ranges:
I chose X to be 100-200 pips in this example, and Y to be 2-3 days.
Amount of data & prefered characteristics:
Let's use 10 years to evaluate the possible parameters on, and "profit" as prefered characteristic.
The process:
Ok, now before we can trade that system, we make an optimisation on the last 10 years.
That means we backtest every possible parameter-combination for our system and choose the best in terms of "profit".
For sake of simplicity, here is a cropped example:
"If the price moved more than 100 pips in the last 2 days, a course correction will happen"
=> 1000$ in the last 10 years
"If the price moved more than 150 pips in the last 2 days, a course correction will happen"
=> 1200$ in the last 10 years
[...]
"If the price moved more than 150 pips in the last 3 days, a course correction will happen"
=> 950$ in the last 10 years
"If the price moved more than 200 pips in the last 3 days, a course correction will happen"
=> 950$ in the last 10 years
Soo, According to our prefered-characteristic (profit), we would choose X = 150pips; Y = 2 days, and then just trade the strategy.
How to ask the right questions
Ok, now that I have described the current process and how it is all done, here comes the "new" part.
The goal itself always stays the same, we want to pick the best parameters (based on some kind of evaluation on the past), and then we want to trade live!
Remember: The only thing a backtest can tell is "How good did my system+parameters perform in the past".
But that is NOT what we want to know! Be sure that you really understand this.
Initial Question; What we actually want to prove with analysis.
"Does the way we choose parameters for live trading ('pick the one with best profit over the last 10 years' in the above example) give us a high probability to pick parameters that are profitable during live trading?"
So we are actually interested in the relationship of past-performance&future-performance, not backtest-results!!
If the answer is No, one or more of the 3 things described in "Initial Situation" are wrong. Might be the logic itself, the parameterranges etc..
If the answer is Yes, the performance in the past and the performance in the future are somehow correlated for our EA, and we can trade the system!
BECAUSE OF A 10.000 CHARACTER LIMIT PER POST; THE SECOND (AND INTERESTING) PART CAN BE FOUND HERE: https://www.mql5.com/en/forum
Ok.. I just noticed that there was something really messed up, and I posted the new thread here in the old one (not once, but as it seems serveral times lol)..
And I can not edit it anymore, so excuse the 5x post, just want to make that clear: My above post will be a new thread, and should not be here.
Moderators: Please delete my above 3 messages (+ this one). Thanks, and sorry for messing up
-Darwin
A WFA takes, for example, the data from 2000 to optimise your system, then tests it on 2001 (which is, from the point of optimisation, the "future" or "live trading").
Then it walks forward and optimises the system on 2001, test it on 2002. Then optimise on 2002, test on 2003 etc.
Do this until you walked through your whole data and only consider the "live trading" in your evaluation.
- DarwinIn my opinion the steps will be as follows,
[/TR]
[/TR]
[/TR]
[/TR]
[/TR]
[/TR]
[TR]
[TD]2000 and 2001[TR]
[TD]2001 and 2002[TR]
[TD]2000-2002[TR]
[TD]2003[TR]
[TD]…….[TR]
[TD]2000-2013Every test will generate a new set of parameters that we can use for the next test, and if the parameters doesn´t make good return for the next one , we should start form the beginning again.
Is that correct , mate ?
Sorry for the late answer!
Well, depending on your chosen in-sample and out-of-sample timespans. Lets make an example with 2 years in-sample and 1 year out-of sample.
Then it would be as following:
1. Optimisation on 2000&2001, test on 2002
2. Optimisation on 2001&2002, test on 2003
3. Optimisation on 2002&2003, test on 2004
4. Optimisation on 2003&2004, test on 2005
5. Optimisation on 2004&2005, test on 2006
6. Optimisation on 2005&2006, test on 2007
[...]
So, optimisation & test timespan always stays the same, and the parameters are only used for one test, not in the next one.
So after optimisation, the best parameterset is tested, test results are recorded, and then the process begins again, with new data & new parametersets.
-Darwin
Might not necessarily be the case. What about a situation when, empirically, one will find out that the 2/1 year is/oos is on the best one but instead a 1/0.5 year gives a better result ? Should you stick to the first at all cost. I think no.
Perhaps it will be true, bactest is a false flag. its different with real..
I recommended to trade in demo contest, it will be the best solution for beginner trader and even for newbie.