EA optimization time and optmization process issue

New comment
 

I am currently working on optimizing an EA based on SuperTrend clustering, and I am honestly struggling to find a solid and reliable optimization process. I have tried different approaches, and the ones that seem the most logical to me involve using genetic optimization with a large set of parameters.

However, when I configure these optimizations in MetaTrader 5, the estimated optimization time often reaches several hundreds or even thousands of hours, which is obviously concerning at first glance. That said, my understanding is that genetic optimization works as a funnel: it does not test all combinations, it progressively eliminates poor candidates, converges toward better regions of the parameter space, and can stop improving once no better solutions are found. In practice, this should mean that the real optimization time can be much shorter than the initial estimate.

My question is therefore the following: is it acceptable or common to see such extremely high estimated optimization times when using genetic algorithms, as long as the parameter space is well defined and the criterion is meaningful? Or is this usually a sign that the optimization setup itself is flawed and should be redesigned?

I would really appreciate any feedback, advice, or experience you are willing to share, whether about optimization strategy, parameter selection, or general best practices when working with SuperTrend-based EAs.

Thank you in advance to anyone who takes the time to help.

 
Be aware, that timers in strategy tester have weird behavior. I once struggle with a similar problem and came to find that timers in the tester are anchored to the OnTick...which means you may get deceiving timing results depending on your tick history.
 
Camilo Mora #:
Be aware, that timers in strategy tester have weird behavior. I once struggle with a similar problem and came to find that timers in the tester are anchored to the OnTick...which means you may get deceiving timing results depending on your tick history.

I really don't see where the OP mentioned an OnTimer() event handler.

In contrast, the OP appears to be talking about a lengthy time of actual backtesting. On that note, I did some research regarding SuperTrend Clustering. If this refers to K-Means machine learning, it appears to be a 5-stage and repetitive process:

  • (1) Decide the number (K) of clusters to identify.
  • (2) Randomly initialize K centroids (cluster centers).
  • (3) Assign each data point to the nearest centroid and associate it with that cluster.
  • (4) Update centroids by computing the average of the associated data points.
  • Repeat steps 3 and 4 until the centroids stabilize.

         (Citation omitted to avoid promotion.)

In this case, I suspect that the culprit is the usual culprit of professional quants and retail algorithmic traders alike... "your code" (quoting Bob the hedge fund manager).

New comment