Optimizing Walk-forward Optimization for Newbies

7 November 2016, 08:59
Stanislav Korotky

WalkForward optimization is very powerful and popular method of EA examination and tuning. Some time ago I have released WalkForward library (WFO) and WalkForward reporter script for MetaTrader. They support several modes and flexible setup, yet they seem a bit complicated for ordinary users. This is why I'm offering you new simplified versions of the library and script. They are completely new products. Below you can find the user guide for the new products. If you're new to walk-forward methodology, you may find some introductory information about it in the user guide for existing WFO products.



WalkForward Light library (wfL) and WalkForward Light report builder (wfLbuilder) form a toolset for simple and rapid analysis of stability of expert advisers.

WalkForward Light library should be embedded into your EA. It monitors optimization process and gathers EA performance data: balance, floating profit, and number of open trades on every bar. WalkForward Light report builder script is then used to generate walk-forward HTML-reports from the data collected by the library.

When you use the library, the built-in tester does not actually perform an optimization, but rather enumerates different combinations of input parameters. Actual optimization is performed inside the report builder when it extracts a specific window from entire period, calculates main performance indicators on this window for different parameter sets (or passes), and finds a set with best results. The optimization criterion is passed to the script via special input parameter.

WalkForward Light toolset implements walk-forward test in an unconventional way. It does not fully reproduce standard walk-forward cycle with continuous in-sample optimization followed by forward out-of-sample testing. This has been changed to address computational and setup complexity of standard walk-forward process (please find details below). If you think you need exact walk-forward testing, you can use more accurate WalkForward library and WalkForward Reporter. The full version requires you to deal with painstaking selection of settings and long-running optimization, which includes several auxiliary parameters added by the library (this can extend optimization space tremendously). If this is not an issue for you, use the full WalkForward library and corresponding reporter. Otherwise, if you want a simple setup as easy as 1-2-3 and fast results, then use wfL and its report builder. Although the light version uses a modification of walk-forward testing, it still provides an adequate estimation of EA robustness.

Please note that the purpose of walk-forward method is not to find optimal EA parameters, but to find best history depth (window size) for further online optimization and how long to keep its results before next re-optimization. The other task is to make sure EA performance does not degrade significantly over time, or, in other words, prevent overfitting.

Here is the main steps of the process (see details about every step below):

1. Embed WFL library into your EA;
2. Run optimization of EA as usual and save optimization results using standard MT4 commands;
3. Copy meta-data collected by the library from tester/Files folder into MQL4/Files folder and run the report builder (this is for MetaTrader 4, version for MetaTrader 5 does this automatically).


The main difference between the simplified and conventional walk-forward approaches is that the former runs built-in optimization (actually, this is just parameter enumeration, as you remember) on the whole testing period at once, not in a step-by-step manner. This introduces a bias in the cases when genetic optimization is used, because parameter sets investigated during the optimization are generally optimal for the whole period, including those successive parts that are actually in "future" with respect to preceding parts. This seamingly spoils conventional walk-forward principle that only data from current optimization window is used for optimization. Yet the generally optimal parameters can give far less profits than locally optimal parameters, which the tester could find if optimization run on a small specific subrange (window) of dates. In a sense, the suboptimal results produced by overall optimization neutralize the effect of looking into the "future". If you still think this is bad, then don't use genetic optimization.

If genetic optimization is not used, the light approach makes no difference, because, in any way, all parameter sets are investigated by the tester.


Embedding the library into EA

To embed the library into your EA use the following header file wfL.mqh, which should be located in MQL4/Includes folder:

#import "WalkForwardLight.ex4"
  int wfl_OnInit(const int cleanUpTimeout);
  void wfl_OnTick();

The library itself - WalkForwardLight.ex4 in case of MetaTrader 4 - should be located in MQL4/Libraries folder (or MQL4/Libraries/Market, in which case the import directive should be #import "Market/WalkForwardLight.ex4").

If after downloading from the Market the library is placed in MQL4/Experts/Market or MQL4/Scripts/Market folder, you need to move it to MQL4/Libraries to make it available for embedding into your expert advisers. Unfortunately, this is how the Market distribution system works by default in case of libraries.

Add the following code into EA:

#include <wfL.mqh>

int OnInit()
  // ... your code

void OnTick()
  // ... your code

wfl_OnInit should be called from OnInit event handler. Parameter cleanUpTimeout specifies a number of seconds after which the library will automatically delete special WFL_-global variables, created and updated by the library during optimization process. The presence of outdated WFL_-global varibales from previous optimization start can lead to incorrect state at next optimization run. The timeout value should be larger than a single tester pass. For example 60 seconds should be OK for most cases. Note that with this setting the automatic deletion of the variables will work only if at least 1 minute has passed since previous optimization run. In any case, you can delete WFL_-prefixed global variables manually before optimization start.

wfl_OnTick should be called from OnTick event handler.

Attention! The file name of your EA should not contain blanks or special characters, because it's used for generation of the subfolder name containing meta-data (see below) and auxiliary global variable name.

Also make sure that work symbol name does not contain special characters as well (such as £, ¥, etc). If it does, you will need to remove the character from the subfolder name manually, and specify the edited variant in the input parameter CSVFolder of the report builder script (see below). Unfortunately there are some limitations in MT platform when it comes to widestring (unicode) characters.


Make some preparations. Switch off the option Skip Useless Results in optimization settings. This can be done once. Clean up Tester/caches folder. This should be done before every optimization with the library. Make sure to utilize Use date flag and From / To fields on the tester's Settings tab as appropriate. Also note that all flags of limitations on the Optimization tab of the Expert Properties dialog should be cleared off before using WFL library.

Run optimization of EA as usual. The optimization period should contain at least 200 bars, but 1000 and above is recommended.

After optimization is finished, open Tester/Files folder and find new subfolder there with a name like this:

<EA name>-<Symbol>-<Timeframe>-<Date>-<Time>

The subfolder should contain csv-files with meta-data stored by the library. The number of the files should be equal to number of passes during optimization. Every record in a file is for a single bar and stores its datetime, balance, floating profit, number of open orders. This information is used lately by the report builder for calculating profits and losses for partial subperiods inside the entire optimization period. Information about separate trades is not collected and not used.

Copy or move the subfolder into MQL4/Files (MetaTrader 5 does this for you).

Save optimization results using standard MT4 command Save as Report from the context menu. This is used as a reference for optimal parameter sets corresponding to every optimization pass. The library for MetaTrader 4 does not store parameters on its own.  

Building reports

Make sure the subfolder with meta-data (csv-files) generated by the library is moved to MQL4/Files.

Run the script wfLBuilder.

Specify the name of the subfolder with meta-data in CSVFolder input parameter (you can copy & paste it from a file manager). You can leave it empty, if optimization has been just finished and proper folder name is still stored in the WFL_FILE_ global variable.

Specify a method of selecting best parameters of EA in Estimator input parameter. This can be one of the following values: 

wfo_profit - net profit amount,
wfo_sharpe - sharpe ratio,
wfo_pf - profit factor,
wfo_drawdown - inversed drawdown (100 - DD) calculated on balance curve,
wfo_profit_by_drawdown - profit amount divided by drawdown,
wfo_profit_trades_by_drawdown - profit amount multiplied by number of trades and divided by drawdown,
wfo_average - average profit of a single trade


After the script finishes, open generated HTML-report with the name corresponding to the given subfolder.

Walk-Forward Cluster Report with overall performance indicators

The HTML-report contains overview tables with Annualized profit/loss, Effeciency and Consistency of walk-forward tests for all combinations of different in-sample window sizes and out-of-sample forward step sizes. These tables are also known as Cluster report. The sizes of in-sample optimization window vary from 10% to 50% of the entire testing period, with step 10%. The sizes of out-of-sample forward step vary from 5% to 30% of the window, with step 5%. For example, if entire testing period compises 1000 bars, then in-sample window will vary from 100 to 500 bars with step 100, and forward step will vary from 5 to 30 bars for the window of size 100, and from 25 to 150 bars for the window of size 500.

The overview tables highlight most successfull settings (window/step size pair) in different intesity of green.

Below the overview tables, detailed walk-forward reports for every combination of window size and step size follow. Every cell in the overview table is a link, by clicking which you'll jump directly to corresponding detailed report.

Walk-Forward Detailed Report with cumulative performance

The detailed table contains date ranges and performance indicators for every in-sample window and correponding out-of-sample test. In addition, it shows overall performance for entire aggregated period of forward steps with such values as profit factor, net profit, and drawdown.  

Every row in the detailed table corresponds to specific pass performed during optimization. The pass number is shown in the rightmost column. User should consult with presaved optimization report (mentioned above) in order to get corresponding parameters (version for MetaTrader 5 shows parameters next to the pass number). If optimization results tab of the tester remains open, you can select specific parameter set by double clicking the row with required pass number in the table with optimization results (in the tester).

The pass number is found for every in-sample region by the reporter script in the following way. It calculates performance estimator selected in the Estimator input parameter and chooses best value. For example, the entire optimization period is 500 bars, optimization window size is 100 bars, and forward step is 20 bars. Then the script will calculate profits and losses for the first 100 bars for every pass or csv-file (which corresponds to specific parameter set), find the best parameters, and calculate profits and losses on the next 20 forward bars (for the same pass/parameter set). This will form the 1-st row in the detailed table. Then the script moves 100 bars window by 20 bars right and calculates profits and losses in the range of bars 20-120, finds best pass (parameter set) for it and checks profits and losses during next 20 forward bars. This will form the 2-nd row in the detailed table. This process proceeds for all possible forward steps.

Please note that building report from a thousand of csv-files may take a while. Recommended timeframes - H1 and larger. 

Performance indicators quick reference

Annualized profit/loss is a hypothetical profit (or loss) which EA could earn in a year if optimization period profit or forward period profit are re-scaled proportionally to their length.

Efficiency is a ratio of annualized profits of forward and back periods.

Consistency is a fraction of profitable forward passes in aggregated forward period.

Share it with friends: