Code Optimization: Multi-Symbol & 3 Indicators

New comment
 

I've learned mql5 iteratively as I've been building my system. In the process, gotten better about abstraction, functional/oop design, using structures and arrays, reusing vars, constants, etc. My issue now is optimizing my code. It runs fine live since single thread performance isn't that big of a deal. But backtesting is a nightmare.

  • System is multi-symbol (38) and uses 3 indicators.
  • PositionManager() is the main loop function 
    • Cycles through positions -> get position data -> decision tree
    • tried my best to clean that up but that massive For Loop seems inevitable
  • Profiler, CopyBuffer() from those 3 indicators is killing my performance. 
    • Even on a single symbol, the constant cycling is intensive.
    • Indicators used in both Entry & PositionManager()
      • Symbol changes often. Handle/Buffer changes often
      • PositionManager() loop forces it to change often.

I know how I might solve this as a single symbol system, just keeping the Handle and it's buffer and only updating as needed, i.e. OnCalculate() or OnTrade().

One change for PositionManager(): if symbol == prev_symbol don't update indicator values. That will save some compute.

But how would I solve this as a multi-symbol system? I can't imagine a persistent 38 handles & buffers being more efficient.

 
percussive21:

I've learned mql5 iteratively as I've been building my system. In the process, gotten better about abstraction, functional/oop design, using structures and arrays, reusing vars, constants, etc. My issue now is optimizing my code. It runs fine live since single thread performance isn't that big of a deal. But backtesting is a nightmare.

  • System is multi-symbol (38) and uses 3 indicators.
  • PositionManager() is the main loop function 
    • Cycles through positions -> get position data -> decision tree
    • tried my best to clean that up but that massive For Loop seems inevitable
  • Profiler, CopyBuffer() from those 3 indicators is killing my performance. 
    • Even on a single symbol, the constant cycling is intensive.
    • Indicators used in both Entry & PositionManager()
      • Symbol changes often. Handle/Buffer changes often
      • PositionManager() loop forces it to change often.

I know how I might solve this as a single symbol system, just keeping the Handle and it's buffer and only updating as needed, i.e. OnCalculate() or OnTrade().

One change for PositionManager(): if symbol == prev_symbol don't update indicator values. That will save some compute.

But how would I solve this as a multi-symbol system? I can't imagine a persistent 38 handles & buffers being more efficient.

Use open price modeling and modify your strategy so that it uses "open prices" instead of "close".

and make sure to call only the data that your strategy requires but not all the data from the indicators themselves.

Avoid using libraries and/or indicators with "icustom", make sure that all the calculation you need is within the same expert advisor in the same way.

If you are using a multisymbol you are surely not using OnTick, right?

also that there are no unnecessary for loops.

 
Manuel Espinosa #:

Use open price modeling and modify your strategy so that it uses "open prices" instead of "close".

and make sure to call only the data that your strategy requires but not all the data from the indicators themselves.

Avoid using libraries and/or indicators with "icustom", make sure that all the calculation you need is within the same expert advisor in the same way.

If you are using a multisymbol you are surely not using OnTick, right?

also that there are no unnecessary for loops.


I'm using OnTick() in strategy tester because it's faster, but live I'm using OnTimer(). Live performance isn't an issue since it's one instance.

I ended up making the "if(symbol == prev_symbol) don't update;" change. Seemed to have improved performance by ALOT. 

Previously I was getting 10% CPU 100% RAM and crashing, on anything more than 2 cores. Now I'm getting 92% CPU on all cores and still able run other applications. I need to double check that the change hasn't introduced any microbugs.

New comment