6-year backtest looks good, and even better with timefilter
Good chance of it. Consider that you've essentially created 5 * 24 binary switches. That's 120 inputs to your system, even if you've formatted them as only 5 inputs. You can do a whole lotta curve-fitting with 120 inputs.
Try running the optimization over 4 or 5 years of data, tuning only on that data. Then run the EA "out of sample" on the remaining 1 or 2 years. See how well it holds up.
Attached test results from my latest EA. Looks good, and with a time filter it looks dramatically better. But...
What I've done is to filter out all non-profitable hours for each of the 5 days... done this over the whole 6-year period, meaning there is a set of trading hours each day of the week (same set every week).
The settings are outlined here, the EA is allowed to open orders only on the indicated hours:
*********
PP="==== Optimizing & Test Settings ====";
OptimizeMode=false;
TradeThisDay= -1;
TradeThisHour= -1;
Monday1Hours="03-04-06-09-10-13-16-20-21-22-23";
Tuesday2Hours="01-02-03-04-05-06-07-08-11-12-14-18-19-20-21-23";
Wednesday3Hours="00-01-02-04-05-06-07-10-11-12-13-15-16-17-18-19-20-21-23";
Tursday4Hours="02-04-05-07-08-10-11-12-13-15-16-17-18-19-20-21-23";
Friday5Hours="00-01-02-03-04-05-06-07-08-09-10-13-14-15-16-17-18-19-20-21-22";
*******
Now the question: Is this a smart thing to do ?
If the backtest was done over only a few weeks I'd say the results was statistically insignificant.
But over 6 years there are many days and many hours... would it still be curve fitting ??
The basic strategy itself looks not that bad. Maybe overtraded (in terms of lotsize) but that is fine.
Well, to the optimization: The period of the optimization does not matter, curve fitting even exists if you optimize over the last 20 years. What you have done here is filtering out the biggest hours where you opened the trades with the greatest loss. (over 30% loss in one trade) Of course this seems only a slightly modification but the effect is big. What are the odds that such an even will happen again in that specifig hour/day? If possible i in your situation would try to find a filter which protects some capital in this situation.
Again the strategy looks good (if the system is not curve fitted), trading it with 1/10th of the risk as you used in the test still gives you a very high return.
//z
zzuegg, I hadn't looked closely at his reports, but I agree 100% -- if his hour filter just prevented him from taking the biggest losses, there's no way it's going to trade forward. That's a textbook example of curve-fitting.
I'd say his risk is high, but not excessively so. By my calculations he's risking about 4-7% on most trades. That's certainly well above the "standard wisdom" figure of 1% risk. But if you believe in aggressive-risk models at all, he's in a reasonably safe region.
I calculated the Kelly value for his non-time-filtered trades, and came up with 0.275. That says he would maximize his returns if he risked nearly 1/4 of his account on each trade! Of course, if you know anything about Kelly betting, you know you don't want to trade anywhere NEAR the full Kelly value. 90-95% drawdowns are likely. With 4-7% risk he's trading about 20-25% of full Kelly, and that's an aggressive but reasonable position. (But you have to be prepared to hit 30-40% drawdowns pretty frequently! In his test I see one peak drawdown of 45%, and 5 drawdowns of 30% or more.) I personally trade a system (with real money) at 20% of Kelly, but I've tested and characterized that system extensively. If I was going to trade this system, I might cut the risk in half, to about 10-12% of Kelly.
But first I'd make sure the rest of the system wasn't curve-fitted like the time filter was!
The result looks very good, but I will mention to check what M1 data do you have in your history database.
PP="==== Optimizing & Test Settings ====";
OptimizeMode=false;
TradeThisDay= -1;
TradeThisHour= -1;
Monday1Hours="03-04-06-09-10-13-16-20-21-22-23";
Tuesday2Hours="01-02-03-04-05-06-07-08-11-12-14-18-19-20-21-23";
But over 6 years there are many days and many hours... would it still be curve fitting ??
- By definition it is curve fitting, but that's not a bad thing, over optimizing is. Over optimized means it works fine for the test period and fails for all others. Optimize for 3 years then test the parameters on the other 3 years to see how good they are. See also Testing (Optimization) Technique and Some Criteria for Selection of the Expert Advisor Parameters for a method to find a set of parameters that each year.
- Monday1Hours="03-04-06-09-10-13-16-20-21-22-23";Means you can't use the Strategy Tester to find them. My code uses 1 int for days and 2 doubles for time so they could be optimized in the ST. (I assume the same time range each day, but that could be easily extended.
WHRoeder, what do you mean by "Means you can't use the Strategy Tester to find them." ?
if OptimizeMode=TRUE the daily trading hours are ignored, I set TradeThisDay = 1 (Monday) and let ST scan through TradeThisHour (00-23). This gives me one ST pass for each hour on the selected day.
Then record the best hours, and manually select the next day, and repeat.
The value of -1 means "trade all" (hours or days). So OptimizeMode=TRUE && TradeThisDay= -1 && TradeThisHour= -1 means trade 24/5.
PP="==== Optimizing & Test Settings ====";
OptimizeMode=false;
TradeThisDay= -1; <-- Integer
TradeThisHour= -1; <--Integer
Monday1Hours="03-04-06-09-10-13-16-20-21-22-23"; <-- String
WHRoeder, what do you mean by "Means you can't use the Strategy Tester to find them." ?
Then record the best hours, and manually select the next day, and repeat.You are recording the best settings and are being the optimizer. You need to run 5 * 24 = 120 times to come up with the best date/times for one set of other parameters.
My version lets the Strategy Tester test all combinations, along with the EA's other parameters.
garyfritz, Seems you're into Kelly theory... After you mentioned it I quickly put together a simple Kelly function. Do you see anything wrong with it ?
I get W around 0.8, and R around 1-4, and Kelly is typically 80% (!)... This doesn't appear right...
You're right that's not right!! :) You're not likely to have a Kelly of 0.80 unless you have one mighty impressive system!
In the classical Kelly equation, K% = (bp - q) / b, where b is your W, p is your R, q is (1-p). So (bp - q) / b should be equivalent to your W – [(1 – W) / R].
I suspect you have a scaling problem. If I remember right, you have to scale b = W by the risk. You could do it on a trade-by-trade basis, but it's probably good enough to look at your average risk, and scale your avg wins and avg losses by your average initial trade risk. Except that should cancel out in the calculation of R. Not sure what's up there -- I remember hitting this issue before and I remember you had to scale.
Also, your calculation of R is off, isn't it? I think it should be R = (PositiveSum / PositiveTrades) / (MathAbs(NegativeSum) / NegativeTrades).
I actually don't use the classic Kelly formula in my work. I use an approximation that's simpler to calculate and is close enough for my needs: AverageTrade / AverageWin. In your example above that would be equivalent to ((PositiveSum + MathAbs(NegativeSum)) / (PositiveTrades + NegativeTrades)) / (PositiveSum / PositiveTrades) -- i.e. the average profit of ALL trades, wins and losses, divided by the average profit of all WINNING trades. That works well for me.
Hi,
I have tried your timefilter optimization today. Using the simplest of all strategies: go long when higher then last bar and go short when lower then last bar. (sl at the opposite and tp = 3 times sl)
This are the results after optimizing the timefilter: (11 Years backtest and optimization)
Currently it is Balance optimized, max drawdown is 27.38%.
In conclusion i vote for such timefilters are definately overoptimizations.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Attached test results from my latest EA. Looks good, and with a time filter it looks dramatically better. But...
What I've done is to filter out all non-profitable hours for each of the 5 days... done this over the whole 6-year period, meaning there is a set of trading hours each day of the week (same set every week).
The settings are outlined here, the EA is allowed to open orders only on the indicated hours:
*********
PP="==== Optimizing & Test Settings ====";
OptimizeMode=false;
TradeThisDay= -1;
TradeThisHour= -1;
Monday1Hours="03-04-06-09-10-13-16-20-21-22-23";
Tuesday2Hours="01-02-03-04-05-06-07-08-11-12-14-18-19-20-21-23";
Wednesday3Hours="00-01-02-04-05-06-07-10-11-12-13-15-16-17-18-19-20-21-23";
Tursday4Hours="02-04-05-07-08-10-11-12-13-15-16-17-18-19-20-21-23";
Friday5Hours="00-01-02-03-04-05-06-07-08-09-10-13-14-15-16-17-18-19-20-21-22";
*******
Now the question: Is this a smart thing to do ?
If the backtest was done over only a few weeks I'd say the results was statistically insignificant.
But over 6 years there are many days and many hours... would it still be curve fitting ??