Optimizing Walk-forward Optimization for Newbies

7 November 2016, 08:59
Stanislav Korotky
0
94

Some time ago I have released WalkForward library and WalkForward reporter script for MetaTrader 4. WalkForward optimization is very powerful and popular method of EA examination and tuning. The problem with the products is that they seem a bit complicated for ordinary users. This is why I'm working on new simplified versions of the library and script. They will be completely new products. While they are in beta, I can send their time-limited builds for evaluation to customers of the existing library upon request. Below you can find the user guide for the new products. If you're new to walk-forward methodology, you may find some introductory information about it in the user guide for existing WFO products.

 

Introduction 

WalkForward Light library (wfL) and WalkForward Light report builder (wfLbuilder) form a toolset for simple and rapid analysis of stability of expert advisers.

WalkForward Light library should be embedded into your EA. It monitors optimization process and gathers EA performance data: balance, floating profit, and number of open trades on every bar. WalkForward Light report builder script is then used to generate walk-forward HTML-reports from the data collected by the library.

When you use the library, the built-in tester does not actually perform an optimization, but rather enumerates different combinations of input parameters. Actual optimization is performed inside the report builder when it extracts a specific window from entire period, calculates main performance indicators on this window for different parameter sets (or passes), and finds a set with best results. 

WalkForward Light toolset implements walk-forward test in an unconventional way. It does not fully reproduce standard walk-forward cycle with continuous in-sample optimization followed by forward out-of-sample testing. This has been changed to address computational and setup complexity of standard walk-forward process (please find details below). If you think you need exact walk-forward testing, you can use more accurate WalkForward library and WalkForward Reporter. The full version requires you to deal with painstaking selection of settings and long-running optimization, which includes several auxiliary parameters added by the library (this can extend optimization space tremendously). If this is not an issue for you, use the full WalkForward library and corresponding reporter. Otherwise, if you want a simple setup as easy as 1-2-3 and fast results, then use wfL and its report builder. Although the light version uses a modification of walk-forward testing, it still provides an adequate estimation of EA robustness.

Please note that the purpose of walk-forward method is not to find optimal EA parameters, but to find best history depth (window size) for further online optimization and how long to keep its results before next re-optimization. The other task is to make sure EA performance does not degrade significantly over time, or, in other words, prevent overfitting.

Here is the main steps of the process (see details about every step below):

1. Embed WFL library into your EA;
2. Run optimization of EA as usual and save optimization results using standard MT4 commands;
3. Copy meta-data collected by the library from tester/Files folder into MQL4/Files folder and run the report builder.

 

The main difference between the simplified and conventional walk-forward approaches is that the former runs built-in optimization (actually, this is just parameter enumeration, as you remember) on the whole testing period at once, not in a step-by-step manner. This introduces a bias in the cases when genetic optimization is used, because parameter sets investigated during the optimization are generally optimal for the whole period, including those successive parts that are actually in "future" with respect to preceding parts. This seamingly spoils conventional walk-forward principle that only data from current optimization window is used for optimization. Yet the generally optimal parameters can give far less profits than locally optimal parameters, which the tester could find if optimization run on a small specific subrange (window) of dates. In a sense, the suboptimal results produced by overall optimization neutralize the effect of looking into the "future". If you still think this is bad, then don't use genetic optimization.

If genetic optimization is not used, the light approach makes no difference, because, in any way, all parameter sets are investigated by the tester.

 

Embedding the library into EA

To embed the library into your EA use the following header file wfL.mqh, which should be located in MQL4/Includes folder:

#import "wfL.ex4"
  int wfl_OnInit(const int cleanUpTimeout);
  void wfl_OnTick();
#import
The library itself - wfL.ex4 - should be located in MQL4/Libraries folder.

Add the following code into EA:

#include <wfL.mqh>

int OnInit()
{
  // ... your code
  wfl_OnInit(60);
}

void OnTick()
{
  // ... your code
  wfl_OnTick();
}

wfl_OnInit should be called from OnInit event handler. Parameter cleanUpTimeout specifies a number of seconds after which the library will automatically delete special WFL_-global variables, created and updated by the library during optimization process. The presence of outdated WFL_-global varibales from previous optimization start can lead to incorrect state at next optimization run. The timeout value should be larger than a single tester pass. For example 60 seconds should be OK for most cases. Note that with this setting the automatic deletion of the variables will work only if at least 1 minute has passed since previous optimization run. In any case, you can delete WFL_PASS and WFL_OPTIMIZATION global variables manually before optimization start.

wfl_OnTick should be called from OnTick event handler.



Optimization

Make some preparations. Switch off the option Skip Useless Results in optimization settings. This can be done once. Clean up Tester/caches folder. This should be done before every optimization with the library. Make sure to utilize Use date flag and From / To fields on the tester's Settings tab as appropriate. Also note that all flags of limitations on the Optimization tab of the Expert Properties dialog should be cleared off before using WFL library.

Run optimization of EA as usual. The optimization period should contain at least 200 bars, but 1000 and above is recommended.

After optimization is finished, open Tester/Files folder and find new subfolder there with a name like this:

<EA name>-<Symbol>-<Timeframe>-<Date>-<Time>

The subfolder should contain csv-files with meta-data stored by the library. The number of the files should be equal to number of passes during optimization. Every record in a file is for a single bar and stores its datetime, balance, floating profit, number of open orders. This information is used lately by the report builder for calculating profits and losses for partial subperiods inside the entire optimization period. Information about separate trades is not collected and not used.

Copy or move the subfolder into MQL4/Files.

Save optimization results using standard MT4 command Save as Report from the context menu. This is used as a reference for optimal parameter sets corresponding to every optimization pass. The library does not store parameters on its own.  



Building reports

Make sure the subfolder with meta-data (csv-files) generated by the library is moved to MQL4/Files.

Run the script wfLBuilder.

Specify the name of the subfolder with meta-data in CSVFolder input parameter (you can copy & paste it from a file manager).

Specify a method of selecting best parameters of EA in Estimator input parameter. This can be one of the following values: 

wfo_profit - net profit amount,
wfo_sharpe - sharpe ratio,
wfo_pf - profit factor,
wfo_drawdown - inversed drawdown (100 - DD) calculated on balance curve,
wfo_profit_by_drawdown - profit amount divided by drawdown,
wfo_profit_trades_by_drawdown - profit amount multiplied by number of trades and divided by drawdown,
wfo_average - average profit of a single trade

 

After the script finishes, open generated HTML-report with the name corresponding to the given subfolder.

Walk-Forward Cluster Report with overall performance indicators

The HTML-report contains overview tables with Annualized profit/loss, Effeciency and Consistency of walk-forward tests for all combinations of different in-sample window sizes and out-of-sample forward step sizes. These tables are also known as Cluster report. The sizes of in-sample optimization window vary from 10% to 50% of the entire testing period, with step 10%. The sizes of out-of-sample forward step vary from 5% to 30% of the window, with step 5%. For example, if entire testing period compises 1000 bars, then in-sample window will vary from 100 to 500 bars with step 100, and forward step will vary from 5 to 30 bars for the window of size 100, and from 25 to 150 bars for the window of size 500.

The overview tables highlight most successfull settings (window/step size pair) in different intesity of green.

Below the overview tables, detailed walk-forward reports for every combination of window size and step size follow. Every cell in the overview table is a link, by clicking which you'll jump directly to corresponding detailed report.

Walk-Forward Detailed Report with cumulative performance


The detailed table contains date ranges and performance indicators for every in-sample window and correponding out-of-sample test. In addition, it shows overall performance for entire aggregated period of forward steps with such values as profit factor, net profit, and drawdown.  

Every row in the detailed table corresponds to specific pass performed during optimization. The pass number is shown in the rightmost column. User should consult with presaved optimization report (mentioned above) in order to get corresponding parameters. If optimization results tab of the tester remains open, you can select specific parameter set by double clicking the row with required pass number in the table with optimization results (in the tester).

The pass number is found for every in-sample region by the reporter script in the following way. It calculates performance estimator selected in the Estimator input parameter and chooses best value. For example, the entire optimization period is 500 bars, optimization window size is 100 bars, and forward step is 20 bars. Then the script will calculate profits and losses for the first 100 bars for every pass or csv-file (which corresponds to specific parameter set), find the best parameters, and calculate profits and losses on the next 20 forward bars (for the same pass/parameter set). This will form the 1-st row in the detailed table. Then the script moves 100 bars window by 20 bars right and calculates profits and losses in the range of bars 20-120, finds best pass (parameter set) for it and checks profits and losses during next 20 forward bars. This will form the 2-nd row in the detailed table. This process proceeds for all possible forward steps.

Please note that building report from a thousand of csv-files may take a while.   


Performance indicators quick reference

Annualized profit/loss is a hypothetical profit (or loss) which EA could earn in a year if optimization period profit or forward period profit are re-scaled proportionally to their length.

Efficiency is a ratio of annualized profits of forward and back periods.

Consistency is a fraction of profitable forward passes in aggregated forward period.


Share it with friends: