How to Improve Optimization Speed in MetaTrader 5 Strategy Tester?

 

Hello traders and developers,

I’ve been experimenting with several Expert Advisors recently and noticed that optimization time in the MetaTrader 5 Strategy Tester varies significantly depending on the settings and number of inputs. I’m looking for some practical tips to improve optimization performance without compromising accuracy.

Here are a few things I’ve tried so far:

  • Using the Genetic Algorithm instead of a full search

  • Running tests on a dedicated VPS with sufficient CPU cores

  • Reducing the data range to the most recent market period

Still, for complex EAs, the optimization takes hours or even days.

I’d really appreciate advice from experienced MQL5 developers —
👉 What are your best practices for speeding up optimization?
👉 Are there specific parameters, hardware setups, or coding techniques that help most?

Thanks in advance to everyone who shares their experience!

Jason Reed

 
Your topic has been moved to the section: Expert Advisors and Automated Trading
Please consider which section is most appropriate — https://www.mql5.com/en/forum/172166/page6#comment_49114893
 
Jason Reed:

I’ve been experimenting with several Expert Advisors recently and noticed that optimization time in the MetaTrader 5 Strategy Tester varies significantly depending on the settings and number of inputs. I’m looking for some practical tips to improve optimization performance without compromising accuracy.

Here are a few things I’ve tried so far:

  • Using the Genetic Algorithm instead of a full search

  • Running tests on a dedicated VPS with sufficient CPU cores

  • Reducing the data range to the most recent market period

Still, for complex EAs, the optimization takes hours or even days.

I’d really appreciate advice from experienced MQL5 developers —
👉 What are your best practices for speeding up optimization?
👉 Are there specific parameters, hardware setups, or coding techniques that help most?

This question has been discussed on the forums many times, and also covered in the algotrading book. Here are most relevant approaches (including something already mentioned by you):

  1. revise your trading strategy to run against bars or timer intervals (then per tick processing [of signals] is not needed, only trailing) - this assumes that accuracy is not compromised;
  2. eliminate indicators, if possible, and transfer all analytical algorithms to the expert itself (indicator calls imply overheads of time and resources);
  3. exclude work with graphics and objects;
  4. cache calculations if possible;
  5. (during optimization) minimize the number of parameters and their combinations, reduce the depth of history (the market changes, sometimes there is no point in digging too deep);
  6. upgrade hardware - add more memory and cores, use fastest disks (SSD for MT5, ramdrive for temp files and logs);
  7. use networks farms or connect to MQ cloud;
  8. use the built-in profiler, find bottlenecks in the code and refactor (rewrite it more efficiently);

Some users even redirect agents' logs to junction points on a virtual drive and then unmount it[ru] so that any logging is eliminated which is supposedly speed up the process.

In the book, one more approach is presented - a custom symbol with special tick compression (making them sparse) without affecting distinct price movements - this is an alternative option if you can't abandon ticks as per the 1-st point above.
MQL5 Book: Advanced language tools / Custom symbols / Adding, replacing, and removing ticks
MQL5 Book: Advanced language tools / Custom symbols / Adding, replacing, and removing ticks
  • www.mql5.com
The MQL5 API allows you to generate the history of a custom symbol not only at the bar level but also at the tick level. Thus, it is possible to...
 
Stanislav Korotky #:
eliminate indicators, if possible, and transfer all analytical algorithms to the expert itself (indicator calls imply overheads of time and resources);

Do you have data about that ? I am very skeptical.

But I suppose it "depends" of the indicator and context. My experience is : it's not true, the measures I did were not conclusive but of course I tried only with some indicators I was needing to use.

 
Alain Verleyen #Do you have data about that ? I am very skeptical. But I suppose it "depends" of the indicator and context. My experience is : it's not true, the measures I did were not conclusive but of course I tried only with some indicators I was needing to use.

In my own experiments, when indicators are based on EMAs (RSI, MACD, etc.), then by using incremental calculations in the EA, it drastically reduced the overhead of the Indicator usage, both in RAM and CPU.

For other indicators that based on SMAs, I found that it depends on the period length and how the indicator's are implemented.

For other types, I don't know as I have not run tests on those cases.

 
Fernando Carreiro #:

In my own experiments, when indicators are based on EMAs (RSI, MACD, etc.), then by using incremental calculations in the EA, it drastically reduced the overhead of the Indicator usage, both in RAM and CPU.

For other indicators that based on SMAs, I found that it depends on the period length and how the indicator's are implemented.

For other types, I don't know as I have not run tests on those cases.

Thanks. My own tests were using proprietary indicators a lot heavier than standard ones like you did.
 
Fernando Carreiro #:

In my own experiments, when indicators are based on EMAs (RSI, MACD, etc.), then by using incremental calculations in the EA, it drastically reduced the overhead of the Indicator usage, both in RAM and CPU.

For other indicators that based on SMAs, I found that it depends on the period length and how the indicator's are implemented.

For other types, I don't know as I have not run tests on those cases.

I can vouch for ATR and SMA calculations embedded in a custom function within an EA. Lightning fast--especially in an MTF context. Mladen Rakic turned me onto this.

This assumes that we're not drawing anything on the chart, of course.

 
Alain Verleyen #:

Do you have data about that ? I am very skeptical.

But I suppose it "depends" of the indicator and context. My experience is : it's not true, the measures I did were not conclusive but of course I tried only with some indicators I was needing to use.

It seems we both have already discussed this matter (probably at the times of MT4 where iCustom was used for reading data from indicators, but MT5 is even more heavier in the sense of multithreading and syncing). I can't produce a ready-made test right now, but I suppose each call to CopyBuffer incurs overheads (measurable in ms) compared to direct access to a local array in EA.

My point was not about specific indicator efficiency (because if a developer transfers the same algorithm from indicator to EA - it would not become more efficient), but about the fact that EA and indicator interconnection requires a bit more resources.

 
Stanislav Korotky #:

It seems we both have already discussed this matter (probably at the times of MT4 where iCustom was used for reading data from indicators, but MT5 is even more heavier in the sense of multithreading and syncing). I can't produce a ready-made test right now, but I suppose each call to CopyBuffer incurs overheads (measurable in ms) compared to direct access to a local array in EA.

My point was not about specific indicator efficiency (because if a developer transfers the same algorithm from indicator to EA - it would not become more efficient), but about the fact that EA and indicator interconnection requires a bit more resources.

More resources without a doubt. 

Worst performance (speed) it depends. 

 
Alain Verleyen #:

More resources without a doubt. 

Worst performance (speed) it depends. 

I suppose you're referring to a well-known principle of exchanging resources for speed and vice versa (either one can be improved at the expense of another). But taking into account that indicator adds more auxiliary tasks for syncing and reading data, the principle is not applicable here, IMHO. Just imagine that a single tick event should be routed to 2 programs instead of 1 (a standalone EA), and then copy some data (instead of accessing local array inplace).

 
Stanislav Korotky #:
Just imagine that a single tick event should be routed to 2 programs instead of 1...
And in 2 CPU threads instead of 1.