Walk Forward Optimisation

New comment
 

Hi everyone,

In my journey in developming a profitable system, I have found some Darwinex video quite helpful for learning about the dangers of over-optimisation and the value of walk forward optimisation. The advice given was that the best approach for making a simple system is to keep the strategy as simple as possible, and to only optimise a maximum of two parameters using a walk-forward method to essentially mean that your final backtest is all OOS data.

I wanted to test this using a simple MA crossover strategy, where there would be no stop losses or take profits, and positions would simply be closed upon receiving an opposite signal. This is shown below using red bars to indicate a short position, and blue bars to indicate a long position.



As mentioned, I only wanted to optimisation the 2 Moving Average periods to keep the the parameter count to 2, as suggested, and I would use the following Walk-Forward plan:


Using this schedule gives a 4:1 ratio between walk-forward optimisation and OOS backtest, and gives 10 years of OOS backtest data.

Next, I wrote the simple strategy on MQL5 and ran the walk-forward optimisation test using these dates, with Open Prices Only modelling as my code operates once per new bar. The MA 1 would be tested from 25 to 100 at a step of 5, and MA 2 would be tested from 50 to 200 at a step of 10.

After running these 11 optimisation runs, I then had the parameters for 10 years worth of out-of-sample backtest data:

Year MA 1  MA 2
2012
 75 90
2013 80 170
2014 95 90
2015 40 150
2016 45 130
2017 95 150
2018 95 80
2019 35 50
2020 50 90
2021 70 50

To be able to run this as one backtest, I added the following section within my code to change the MA parameters depending on the year:

   MqlDateTime STime;
   datetime time_current=TimeCurrent();
   datetime time_local=TimeLocal();
   
   TimeToStruct(time_current,STime);
   int year = STime.year;
   
   int ma_1_period_optimised = 50,
       ma_2_period_optimised = 200;
   
   switch(year)
     {
      case  2012:
         ma_1_period_optimised = 75;
         ma_2_period_optimised = 90;
     
      case  2013:
         ma_1_period_optimised = 80;
         ma_2_period_optimised = 170;
     
      case  2014:
         ma_1_period_optimised = 95;
         ma_2_period_optimised = 90;
         break;
         
      case  2015:
         ma_1_period_optimised = 40;
         ma_2_period_optimised = 150;
         break;
     
      case  2016:
         ma_1_period_optimised = 45;
         ma_2_period_optimised = 130;
         break;
         
      case  2017:
         ma_1_period_optimised = 95;
         ma_2_period_optimised = 150;
         break;
         
      case  2018:
         ma_1_period_optimised = 95;
         ma_2_period_optimised = 80;
         break;
         
      case  2019:
         ma_1_period_optimised = 35;
         ma_2_period_optimised = 50;
         break;
         
      case  2020:
         ma_1_period_optimised = 50;
         ma_2_period_optimised = 90;
         break;
         
      case  2021:
         ma_1_period_optimised = 70;
         ma_2_period_optimised = 50;
         break;
     }
   
   if(use_optimised_parameters){ ma_1_period = ma_1_period_optimised; ma_2_period = ma_2_period_optimised;}
   if(!use_optimised_parameters){ ma_1_period = ma_1_period_unoptimised; ma_2_period = ma_2_period_unoptimised; }


Then I ran a backtest using Every tick based on real ticks, from 2012 to 2021 using delays and $3 round turn lot comission:




Whilst this isn't the best equity curve and does go through periods of loss, it does show the benefit of walk-forward optimisaiton.

My question is; would this be sufficient to run this strategy live in 2022 based on continuing this process by running an optimisation from 2017-2021 to get the parameters?

 

can you share the codes for the walk forward optimization? 

Incubation period is important to redeem from your feelings to your system that happens to occur because of the time and effort you spent on it.

But Jack Schwager offers a method to save time: choose an early period like early 2000's for a second walk forward era.

At those times the spreads were wide, HFT wasn't active so that using this period would be a good tackle for your system's confirmation.

 
newin #:

can you share the codes for the walk forward optimization?

Try this.

Validate
Validate
  • www.mql5.com
Проверка торговых советников
 
fxsaber #:

Try this.

I will.. thank you.

 
Benjamin David Hardman:

My question is; would this be sufficient to run this strategy live in 2022 based on continuing this process by running an optimisation from 2017-2021 to get the parameters?

Backtests are misleading nearly 100% of the time. They can tell you that your system behaves as intended but it's really hard to do any optimization with them without introducing hindsight biais into your decision making. I know Darwinex spends a lot of time telling you how to reduce this but don't forget that the main purpose for their videos is to promote their business. I personally believe you shouldn't even run a backtest until you know you have a winning system, and if you do, I honestly think the risk of screwing up your system with an optimization is probably greater than your odds of improving it. Maybe I'm wrong, but I believe algo trading is best used when the goal is to automate your trading process and that using it to come up with "winning parameters" is a fools errand.

"Hindsight bias, also known as the knew-it-all-along phenomenon[1] or creeping determinism,[2] is the common tendency for people to perceive past events as having been more predictable than they were."

https://en.wikipedia.org/wiki/Hindsight_bias

 

FYI you aren't really following the best practices that they recommend. If you're going to follow their advice, you're not supposed to use more than 2-3 steps for each parameter you optimize but you're using at least 31! (50 to 200 with steps of 5). What they teach is that you should at the most be doing something like 100-200 with steps of 50. Also they tell you that to avoid over optimizing you shouldn't really optimize two parameters for your entries which is what you're doing. So you are using something like 961 possible parameter combos for your entries when they tell you to use no more than 3 degrees of freedom in your optimization for each part of your system. And at the most, they usually recommend optimizing no more than one parameter for your entry, one for your filter, and one for your exit. So based on this alone, this will fail in live testing because of major curve fitting issues. You're doing all the things that Darwinex tells you not to do. Your optimization is going to be able to give you a nice PNL because it has too many degrees of freedom, that's not a good thing, unless you're trying to build a robot that's only good in a backtest but we all know those are scams. Walkforward optimizations are a good tool to use if you want to optimize your system, but it won't help you if you don't follow optimization best practices.

*edit: you also don't have enough trades to get much statistical significance. As per the same source, I think if I recall correcly the edge only starts to show with some degree of consistency once you're passed 5k+ trades. I may be wrong but I know their main guy usually gets 17k+ trades in a backtest in order to make sure it's statistically significant and ran some simulations to prove that anything below 1000k is basically not going to give you anything of value. I can also tell you from experience that anything under 1000k usually doesn't work. So I mean again, if you're going to try to use this source in order to learn about optimization best practices, I suggest you keep watching more videos because there's a lot of important principles that you are not implementing right now.

 
fxsaber #:

Try this.

Hi, I uploaded the include .mqh files and validate EA .

That's what I see (attachment)

There is nothing on it. How will I arrange the periods and how to make it start?


and an additional question: is Validate EA suitable for  ' complete optimization ' ?

and second question: can I use it with 1-minute OHLC mode?

thank you in advance

 
newin #:

Hi, I uploaded the include .mqh files and validate EA .

That's what I see (attachment)

There is nothing on it. How will I arrange the periods and how to make it start?


and an additional question: is Validate EA suitable for  ' complete optimization ' ?

and second question: can I use it with 1-minute OHLC mode?

thank you in advance

 
fxsaber #:

Hi fxsaber,


Thank you very much.

This is a very useful EA.


I have a few questions and a few comments below, please reply when you have time.


my questions and comments are:

1. what is inmaxpasses ,inparameter, outsamplesymbol?

2. I want to continue testing with 1minute OHLC, is it doable? ( tick mode takes huge time)

3.It chooses 'best setting' to do single backtest as forward testing time interval, what is the criteria for 'best setting'?

it would be better to have the forward results as it is regularly produced by strategy tester or an excel file can be produced for it( including maxdd,PF,RF,sharpe...etc, as strategy tester already produces for forward test). So I can choose 'best setting' for forward test.


as I said, it is beautiful. I appreciate for your work.

 
newin #:

1. what is inmaxpasses ,inparameter, outsamplesymbol?

This parameter limits the number of passes during optimization.

2. I want to continue testing with 1minute OHLC, is it doable? ( tick mode takes huge time)

Yes.

3.It chooses 'best setting' to do single backtest as forward testing time interval, what is the criteria for 'best setting'?

The best result for the selected criterion during optimization.
 

2. I want to continue testing with 1minute OHLC, is it doable? ( tick mode takes huge time)

Yes.


How?

it automatically changes my 1 minute OHLC choice to tick mode by itself.

thanks in advance.

12
New comment
Reason: