Select the right optimization parameters?

New comment
 

Dear all,

I am looking for a systematic way of choosing the right parameter set after an optimization.

How would you proceed to do so?

My objective is to optimize on 1 year, then select the parameters, then trade during 3 months, then optimize again on 1 year, etc.... This would allow to include market structure changes in the selected parameters (new players, volatility,...)

So far I have come to this solution (which I am not fully happy with...)

1. Filter all optimization results on some exclusive criteria --> Minimum number of trades, minimum Profit ratio, maximim % drawdown.

2. I take, within all the optimization results, the best parameters (ex: a P/R of 4 is the best, a max number of 300 trades and max 1% of drawdown)

3. I create a corrected profit for each result which is: Profit * ProfitRatio / BestProfitRatio * NumberOfTrades/BestNumberOfTrades * BestDrawdown/Drawdown

This enables to eliminate juicy parameter sets with to much risk, too less stat representativity.

4. Then I take the 25 best corrected profit and calculate the average of the value of each parameter.

I then use this average of for each parameter to test forward the strategy.

This can be easily done in Excel.

How do you guys proceed? I want to find the most stable parameter set... not nessarily the most profitable...

Thanks for your anwsers!!!

See ya.

Pat.

 

Pat the question you need to ask yourself, and should find yourself wanting to answer, is on what basis are you assuming past results are going to be indicative of future results?

Set aside all the fancy filtering methods for the moment. Presumably you are doing all this because you believe, for whatever reason, that once you find a golden set of filtering parameters you will then be able to downselect trade parameters based on their historical performance with expectation of this leading to future results of similar characteristic.

So what is your reasoning? (it is a relevant question because the answer determines whether or not metrics like drawdown and profitratio and bestnumberoftrades actually serve any purpose as filtering metrics for downselecting trade parameters for future usage)

I could tell you in excruciating detail what my specific filtering parameters are, and why, but those analyses will mean nothing to you and your strategy if the assumptions built into your strategy for market prediction are not the same as the ones I used in building mine.

For example, here is a useful article on why backtesting metrics such as drawdown are pretty much useless for capturing/expressing risk: Minimizing your risk of ruin

Some people glance it over and conclude "meh, who cares"...others see a deeper message in the maths.

Also some (including myself) worry about things like MAE and MFE: Mathematics in Trading: How to Estimate Trade Results

And some folks come to realize that their filtering methodology merely resulted in maximizing the coincidence of random events occurring favorably in their direction (a phenomenon that won't likely be repeated in future): My First "Grail"

 
Humm.. Phillip, I take it you don't believe in Bigger_Loss-to-Win $per Trade; say stop-loss 100 to profit 10. Is there any (Bigger_Loss-to-Win $per trade) you believe in; say 5-to-1 or 2-to-1? Now I'm just guessing here, please correct if my belief is misplaced. Me/Myself/Personally don't have a Rule, I gather that there's a mathematical point where 10-to-1 would be acceptable, say you have 11 winners to every 1 loser. Of course I'd prefer a (Bigger_Win-to-Loss $per trade) but don't mind otherwise. I've met people who would just not allow anything less then 2-to-1 (Bigger_Win-to-Loss $per trade). Their thinking is, they have more to win than lose and this is money management to them.
 

The basic issue here is what have you (each of us) done to ensure that our EA's generate the kinds of past results in backtesting which can also be expected to be produced in forward tests.

Many metrics of performance analyses as given by the strategy tester report are dependent on the specific chronology of a past price activity in the timeseries.

Consecutives wins. Drawdown. Etc.

The only time and place the values of these metrics tell us anything about the future performance characteristics of our EA is if in the future our EA encounters a virtually identical timerseries.

I think people tend to gloss over the aspect of analyzing how or why their strategy is forecasting future market events simply for the fact it requires a lot of thinking and effort. Much easier to just press a button, make the computer do the work (or think it is doing the work), and then select the "winningest" parameters and conclude "well with these parameters my EA was able to forecast the market superbly well...how could it not continue doing such for a little while longer?". Which of course is a fallacy trap.

This is not to say that past performance cannot be used to project expectations (with confidence) of future results...but you have to have built your strategy with specific intentions of it being capable of such, and you have to use statistics to have optimized it for such. It is not an intrinsic property of the strategy, it has to be engineered in.

And I'm not claiming to have done it properly myself either, I'm not sitting on the grail here.

New comment
Reason: