Backtesting and optimization are valid instruments?
I'm not experienced with live yet, but getting there. The way you think about it seems similar to me, right now I have two main fears:
1) accuracy of backtest results... MT4's backtester is awful for high-frequency (short timeframe) strategies. Solution: I coded my own in a script.
2) curve fitting... are your optimisations only good for the time period your tested? Solution: split your history, optimise on one part, and then test on the other part.
If you can, do a parameter sweep. Looking at your best set of parameters, does a minor change throw off the result completely?
Also, kind of a hack way I'm using at the moment, is to test 2 days or more days for each parameter set and see if there isa correlation. It's painful stuff as it kills all the enthusiasm you had with a 1 time period test.
Here you can see params on the left, then results for 2 days on the right. Excel conditional formatting makes it a little easier to spot patterns. e.g this one's kind of encouraging as the net profit seems to correlate and the parameters at the top seem to be around the same values:
If you can, do a parameter sweep.
I'm not experienced with live yet, but getting there. The way you think about it seems similar to me, right now I have two main fears:
1) accuracy of backtest results... MT4's backtester is awful for high-frequency (short timeframe) strategies. Solution: I coded my own in a script.
2) curve fitting... are your optimisations only good for the time period your tested? Solution: split your history, optimise on one part, and then test on the other part.
If you can, do a parameter sweep. Looking at your best set of parameters, does a minor change throw off the result completely?
Also, kind of a hack way I'm using at the moment, is to test 2 days or more days for each parameter set and see if there isa correlation. It's painful stuff as it kills all the enthusiasm you had with a 1 time period test.
Here you can see params on the left, then results for 2 days on the right. Excel conditional formatting makes it a little easier to spot patterns. e.g this one's kind of encouraging as the net profit seems to correlate and the parameters at the top seem to be around the same values:
Regarding point 1 I don't know how long it takes in terms of time and effort to get super-accurate 1min or tick chart since my EAs are design to work on 1hr or 1day timeframe. For point 2 I believe you are speaking about "walkforward": I tried the sw Walk Forward Analyzer for MetaTrader 4 but the issue is that I have too much parameters to optimize...it takes forever. I'm not lazy here but fx is only a hobby for me cannot spend too much time on it and, again, is it worth to spend hours in optimizations?
I'm curious to hear more about this parameter sweep, what do you mean exactly? Just try different set of parameters on the same EA?
It depends on the strategy. If good parameters give profit and bad parameters give loss .. then yes, optimising is worth it. Then again, you can't know that without backtesting.
If you gues parameter A has a min value of 1 and max value of 2, a parameter test is running the same test on the same data using B=1, B=1.1, B=1.2 etc. Or more safely, B=0.5 to B=2.5. If B=1.7 gives great results but B=1.75 and B=1.65 are awful, you might conclude the good results are from data fitting.. look more closely at the prices and trades.
Good walk forward testing is more complex than what I'm doing, it has an optimisation+testing, followed by another overlapping optimisation+testing etc.
It shouldn't take that long. Currently my backtest on takes 1 seconds to backtest a day's tick data for one iteration (one parameter set), so with for loops, I leave it parameter sweeping over night and get me results in the morning.
Over optimising and backtesting can be time consuming and not as rewarding in the end if you're not very careful.
As a 'rule' the only dependable EAs are those that make their entries/ trades based on volatility. Anything else is very suspect.
Over optimising and backtesting can be time consuming and not as rewarding in the end if you're not very careful.
As a 'rule' the only dependable EAs are those that make their entries/ trades based on volatility. Anything else is very suspect.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
I'm quite new to the fx world and I'm try to learn and experience as much as possible. I developed a set of EA which I backtested (I used Alpari data) and optimized for several pairs. The results of these simulation are quite good but, from a pure mathematic point of view, how accurate and reliable are data coming from backtesting and optimization?
The market prices's changes are caused by several causes and many of them are fundamental-related. Moreover, when I'm optimizing an EA, how can I trust that I'm really getting optimized parameters and not only curve-fitted parameters? In fact there is always a number of parameters which lead the EA to be super-profitable.
I'm writing this post since, as I said, I'm approaching to this world and before spending a lot of time and effort optimizing and backtesting EA just to see them failing miserably in demo (or live!) I would like to have the opinion of expert and experienced guys who already faced my issues.
Thanks for your help.