From Optimisation to Validation: Should EA development be based on a Statistical Framework?
I’m interested in hearing how others here approach these points:
-
Do you select parameters based on peak performance?
-
Do you intentionally test under worse-than-average execution conditions?
-
Have you observed strategies that survived because they were “under-optimised”?
-
How do you handle recalibration without overfitting?
Looking forward to a technical discussion and learning from others!
Apologies this is my first forum and will take the time to re-write it.
I guess the question is, however, if the content is valid or not?
English not my first language, so I had it translated.
But let me observe that when we do math calculations no one is asking if these have been done using a calculator or if these have been done with pen and paper !
Given current developments we should probably consider progress beyond "who wrote it" and move towards "what it has been written" ...
Shall I remove and repost my human based English version?
Thanks for assistance.
Apologies this is my first forum and will take the time to re-write it.
I guess the question is, however, if the content is valid or not?
English not my first language, so I had it translated.
But let me observe that when we do math calculations no one is asking if these have been done using a calculator or if these have been done with pen and paper !
Given current developments we should probably consider progress beyond "who wrote it" and move towards "what it has been written" ...
Shall I remove and repost my human based English version?
Thanks for assistance.
It would be great to see K-fold validation during built-in optimization in MT5, as well as Monte-Carlo estimators (at least by means of resulting trade order permutation).
MQ thinks other ways of MT5 developement though.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Why Robust EAs Are Built on Statistical Validation, Not Optimal Parameters
Over the years I have been doing trading on my own and buying some EAs in the market, I’ve noticed that most discussions around EA development — here and elsewhere — still revolve around the same core objective:
Usually this means:
Optimising in-sample
Selecting the peak performer
Hoping that out-of-sample / live trading confirms it (sometimes even constantly fine-tune parameter choice)
Sometimes it does. Often it doesn’t — especially once realistic execution costs and live trading conditions enter the picture.
I’d like to outline a different framework, one that treats EA development and systematic trading as a statistical validation problem, not an optimisation contest.
The goal is not to maximise historical profit, but to maximise survivability and durability under changing market and execution conditions.
Optimisation vs validation: a conceptual shift
Traditional approach: Optimisation → pick the best parameters → validate
Statistical approach: Define acceptable behaviour → validate across dimensions → select stable regions
In other words, instead of asking: Which parameters performed best?
we ask: Under which conditions does the strategy remain statistically viable?
This sounds subtle, but it fundamentally changes how strategies are built and selected. The mathematical problem becomes one of stability rather than maximization.
Why testing under high-cost assumptions is intentional and needed
One of the most controversial parts of this approach is deliberately testing under conservative assumptions, such as:
Wider spreads than typical
Slippage closer to live reality
Execution filters that reject marginal trades
Yes, this reduces profitability as we all know But that is exactly the point.
High-cost assumptions act as a stress test:
They remove edges that only exist under ideal conditions
They penalise over-trading
They expose parameter sensitivity early
If a strategy only works when:
spreads are tight,
execution is perfect,
and fills are assumed instantaneous,
then the strategy is not robust — we would say, lending from probability theory, that the strategy is conditional rather than unbiased.
A statistically robust profitability assessment should be unbiased
From re-optimisation to calibration
Another key implication of this framework is how strategies are maintained.
Instead of:
frequent re-optimisation,
parameter chasing,
resetting logic every few months,
the focus shifts to:
monitoring performance drift,
checking whether results remain within expected statistical bands,
recalibrating within pre-validated regions when needed.
This preserves:
the original edge,
the statistical structure,
and the risk assumptions.
Calibration adjusts the strategy without redefining it.
What a statistical approach trades off — and ... is that’s acceptable?
Let’s be honest: this framework does not produce:
the highest backtest profit,
the smoothest equity curve,
or the most impressive screenshots.
What it does produce is:
lower fragility,
fewer unpleasant surprises live,
and behaviour that matches expectations more often than not.
A fully controlled drawdown pattern.
In my experience, this trade-off is not a weakness — it is a design choice.
Open questions for discussion
I’m interested in hearing how others here approach these points:
Do you select parameters based on peak performance?
Do you intentionally test under worse-than-average execution conditions?
Have you observed strategies that survived because they were “under-optimised”?
How do you handle recalibration without overfitting?
Looking forward to a technical discussion and learning from others!