Preventing overfitting - page 3

New comment
 

Optimization is more art than science.

According to my experience, no single books address this completely. Looked at most of established books in math, statistics, quantitative trading, their procedures are partly agreeable only and many of them are misleading. Even reputable professionals in this area will have different and conflicting opinions.

Went few conference arranged for developers from top 3 IB. Mostly useful and practical ideas and knowledge are only passed from mouth to mouth like apprenticeship. Still optimization is more arts than science in my view.

If you really would like to get some useful information from just text book or article only, I will recommend to read medical statistics journals or geo statistics journals. Amazingly all the technology used in trading and investment was already used by these scientist like 1 or 2 decades ago.

Also forgot to say. LapPack or blast has very good optimization routine written by the smartest scientist ever. Understanding these routine can help you a lot too in investment and trading.

Kind regards.

 
Ubzen:

Nice Article. I agree with RaptorUKfirst comment " Don't optimize at all". Otherwise, you'll run into problems explaining how to properly optimize. Most of us would agree that curve-fitting exists. However, most would also dis-agree upon how to avoid it.

Someone can take a simple SMA_CrossOver Price, create allot of Optimization_Switches for example Periods, BarCounts, Stoploss, TakeProfit, TimeOfDay, etc. Each of those can end up having about 100_steps. Thats 100*5 [ or probably 100 square ] different results with just the few above, and voilà, you have a profitable sma strategy which more than likely wouldn't be profitable in live testing.

There are soo many different material on how-to optimize that then again brings me toward the KISS principal.

  • Like doshursuggests, you can try optimizing with small_range from where you started.
  • Some people might suggest larger steps, instead of stepping by 1, use 10, or 100 etc.
  • Some people might suggest you optimize on a small date_range size then walk-forward on large date_range.
  • Some people might suggest you optimize on a large date_range size then walk-forward on small date_range. 
  • Some people might suggest you optimize on all available data.
  • Some people might suggest you need a large number of trades.

What method someone uses, usually depends on their forex world_view. One of the popular view on optimization is bullet#4 where you optimize for about 4-months, and then run a back-test for the next 2-weeks or so following the optimization end_date. Obviously someone using such method believes that market behaviors are short_live and needs constant optimization to stay on track. And thats just one example of forex world_view.

What I believe about optimization is this [ should don't optimize not work for you ]: I'll have to use an example with blackjack simulators to illustrate. Lets say I wanted to know a good value to take insurance while i was counting cards. So i simulate a couple of million hands of blackjack and the results comes at follows.

0__ 

1_______

2______________ (best count for insurance because it has the longest bar)

3_______ 

4__ 

As you can see, 2 gives the best result because it has the longest ___bar___ or number of wins at that count. But whats important here is that number 1 and 3 neighbors #2 in being second bests. Followed by 0 and 4 on both wings. This idea of a cluster is what give me confidence to select 2 as optimal.

 

However, with most forex optimizations, the results usually don't create that curve shape with a peak.

0__

1______________ 

2__

3______

4______

Clearly number1 creates the best results but this doesn't inspire much confidence from me.

 

To summarize, don't optimize, observe the market or think up a strategy, code it and should work on the first run. Otherwise think up something else. Should it work on the first run, then look into optimizing taking into consideration the cluster effect. Use all available data for testing, accept any weakness and move forward.

Am I on the right walk?

 

My 2 cents to the overfitting subject - a 100% profitable, open-source strategy:

www.tradingview.com/script/XnvfXLPF-NoNonsense-Forex-high-timeframe-trading-absurd-NON-REPAINTING/



123
New comment