Critique my optimization process!
Your optimization approach seems very thorough.
- Be cautious of overfitting, as you mentioned, especially when drilling down to smaller parameter sets. Having a solid forward test period helps mitigate this.
- Consider adding robustness checks, like walk-forward analysis or multi-market/instrument tests.
Overall is excellent. Keep up with good work!
Happy trading!
Your optimization approach seems very thorough.
- Be cautious of overfitting, as you mentioned, especially when drilling down to smaller parameter sets. Having a solid forward test period helps mitigate this.
- Consider adding robustness checks, like walk-forward analysis or multi-market/instrument tests.
Overall is excellent. Keep up with good work!
Happy trading!
awesome thanks for the feedback!
Very fascinating. I like how you move between "genetic" and "full/slow" optimization.
As far as possible and as I imagine anyone would do, I always try to test with "FULL" and I also keep a small balance to use the agents as well. I would echo the comment about over-fitting.
When I try, I divide signal testing (indicators and such) completely from position testing (closing, position modification like SL/TP and such). Also, market profiles where possible if we can better distill these, you HAVE to test in different scenarios because the strategy that yields today, WILL plunder tomorrow when it meets a market profile that you have not prepared and tested for.
Getting good data and getting MQL5 back-testing has been better for me than with MT4 however, there is still MUCH to be desired.
Best regards to you and the crew on here.
I'm not a fan of optimization. In my opinion, if your EA needs optmization to be profitable, it's not a good EA/strategy.
That being said, when I am developing a new EA (a fresh new idea), I choose a couple forex pairs and a time period (let's say the year of 2020, as an example). Then I run all my tests and optimizations on that small time period, just to see if parameter A or B should be higher or lower. But never more than 2 or 3 parameters.
After that I run the EA on random pairs and random periods of time. If it's not profitable on at least 70% or 80% of the tests, it's not a good strategy.
This means that 99% of all EAs I've developed are not "good enough". If I start to optimize every EA on every single pair, 50% would turn profitable...
EDIT:
What I mean is:
If you optimize too much, too many parameters, you are making your EA work on the specific market movements that have already happened.
On the other hand, if you optimize just a couple parameters, to find out if a certain value should be half or double what you are currencly using, that's ok. A good EA is more about a general idea, a new/different fundamental strategy, than a very tailored set of parameters that were optmized for a specific pair.
Thank you for sharing this!
Very fascinating. I like how you move between "genetic" and "full/slow" optimization.
As far as possible and as I imagine anyone would do, I always try to test with "FULL" and I also keep a small balance to use the agents as well. I would echo the comment about over-fitting.
When I try, I divide signal testing (indicators and such) completely from position testing (closing, position modification like SL/TP and such). Also, market profiles where possible if we can better distill these, you HAVE to test in different scenarios because the strategy that yields today, WILL plunder tomorrow when it meets a market profile that you have not prepared and tested for.
Getting good data and getting MQL5 back-testing has been better for me than with MT4 however, there is still MUCH to be desired.
Best regards to you and the crew on here.
Thanks for writing back! yeah there definitely could be a lot more on the optimization side in mt5 i agree. that's an interesting point about market profiles, are we talking like using a moving average for a trend filter kind of thing or? im definitely curious about how you would go about it
I'm not a fan of optimization. In my opinion, if your EA needs optmization to be profitable, it's not a good EA/strategy.
That being said, when I am developing a new EA (a fresh new idea), I choose a couple forex pairs and a time period (let's say the year of 2020, as an example). Then I run all my tests and optimizations on that small time period, just to see if parameter A or B should be higher or lower. But never more than 2 or 3 parameters.
After that I run the EA on random pairs and random periods of time. If it's not profitable on at least 70% or 80% of the tests, it's not a good strategy.
This means that 99% of all EAs I've developed are not "good enough". If I start to optimize every EA on every single pair, 50% would turn profitable...
EDIT:
What I mean is:
If you optimize too much, too many parameters, you are making your EA work on the specific market movements that have already happened.
On the other hand, if you optimize just a couple parameters, to find out if a certain value should be half or double what you are currencly using, that's ok. A good EA is more about a general idea, a new/different fundamental strategy, than a very tailored set of parameters that were optmized for a specific pair.
Yeah i understand what you mean and i think that theory definitely holds a lot of weight, i've tried to no avail with that approach (perhaps just not just hard enough though..) it's definitely an interesting take, you're like way on the other side of the spectrum than i am, it's kind of interesting how everyone has there own way of going about it
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Hi everyone, I'm once again in the optimization rabbit hole and I figured you know what, why not post my own process on the forum and let everyone tear it apart lol, who know's maybe i can learn something from it!
-Don't hold back i promise i can take it lol! (also i realize i can't describe all the nuance things here very well, may have to make a video at some point).
*special note: be it from the algo or my methodology, performance in the last few months using this has been largely flat. (not the holy grail, don't get excited).
Optimization - Single Symbol - For sake of example USDJPY on a Range breakout strategy on the M5 timeframe
- Using custom performance metrics - CAGR(coded into my ontester()). *note: i have found recovery factor produces similar results and sometimes i use that.
-> i have some custom inputs, any results displayed has to have a minimum of 100 trades and < 10% DD
Testing period: 10 years, 1/2 backtest 1/2 forward test, using random delayed and 1:33 for leverage
i trade based on equity and i always leave "RiskPercent" at 0.1
-I'm using open prices and only running the full onTick() on the open of a new bar **note: not using Bars in the code to detect a new bar, using iTime instead. - depending on the strategy usually using M5 and up for timeframe, if it's a scalping strategy then in that case i'll using tick data.
***Diving for direction***
I start with a genetic optimization using a "gazillion" parameters and steps - i call it "diving for direction" (the use of the genetic algo tends to take 1-4 times to find "direction" since there's so many combinations and from there maybe another 3 ->MAX<- to isolate an area a bit (in my opinion it's a bit of a "feel" approach (so i think) as to not overfit the data, hard to quantify..
-> In my head i've kind of formulated and idea that you can take say a (for sake of example) range breakout strategy and trade it in a million ways, so i try to tap into a "pocket" of 1 of thousands of ways to go about it and "work and area".
When i find and area that has a cluster of good performance (looking at result graphs (usually the data and 3D graph) and the backtest and forward test curves are both (for sake of example) at a 45 degree angle (knocking it out of the park) i call that my general direction.
-> at this point i've roughly found my strategy and have it "in the ballpark". - lots left to do at this point.
***Isolating and area***
- the process now switches to doing a few more genetic algorithm optimization runs, each slowly knocking out some of the useless parameters usually i get rid of 3-4 parameters that don't look to be contributing anything worthwhile to the strategy. (either because of steps being used when a boolean is false or just poor performance say a trend filter using a 100SMA, if it's better not to use it, thats one i de-select)
*minimums for selection of positive performing parameters and digging deeper into them: statistical significance (to put a number on it 100 trades a year) and less than 10% DD. i don't tend to look at profit so much, i tend to study the equity curve (i keep risk at 0.1, in my opinion i'd rather find a good equity curve and amp up the risk)
-> i'm just trying to dig a little deeper into this area to isloate this strategy (at this point i realize i have to be careful not to just curve fit, so theres a point where i see im in a broad swath of good performance and i have to say ok, good enough. now lets switch to slow opts)
at this point i've run genetic 1-4 times for direction (4 if the algo isn't very good) and 1-3 times to isolate and area.
***Taking it slow***
i switch to slow optimization and start testing in blocks of 3-4 parameters.
however: if i'm at a point where the genetic has me down to testing less than 30,000 parameters ill just run the whole batch and pick something in the middle with some good performance (back and forward curves, drawdowns within limits)
if i still have a "gazillion" combinations i try to go about it logically.
ill test all the stop loss parameters, find the best general settings, save those
then ill test the best take profit settings , save those
then test the best signal related settings, etc etc.. until i'm all done the parameters.
*** A quick double check***
then ill double check the equity curves, results data (especially making sure it's taking long and short trades, i've noticed sometimes i can fit a little to much and it'll only take long trades for example) and ill run a single and watch for the visual logic to make sure something didn't get messed up and it's a "one-off".
-if i'm having trouble on a particular optimization i'll also study the results of the built in results like sharpe ratio, definitley recover factor i like that one, etc.
then it goes to a demo trial.
if you have any feedback i'd love to hear it, good or bad!