Discussing the article: "Build Self Optmising Expert Advisors in MQL5"

[MetaQuotes]

Check out the new article: Build Self Optmising Expert Advisors in MQL5.

Build Expert Advisors that look forward and adjust themselves to any market. Self-optimizing automated systems are essential in today's dynamic financial markets.

Developing a trading bot that can adjust to current market conditions is key to stable algorithmic trading strategies. Our aim is to go beyond creating narrow-focused bots limited to a few symbols. We intend to design systems with the capacity to learn and adapt to any trading symbol. This guide focuses on using MQL5 to develop bots that can self-optimize to any trading environment.

MQL5 is ideally suited for this task, contrary to some beliefs. Its API provides extensive matrix and vector functions, allowing for the creation of compact machine learning models. This introduction emphasizes using MQL5 for building self-optimizing bots. An Object-Oriented Programming approach reduces repetitive coding and enhances adaptability across different time frames and market conditions.

Opting for MQL5's matrix and vector capabilities over alternatives like ONNX and Python has considerable benefits. Using an ONNX model would require separate model instances for each trading symbol and new models for any minor parameter changes, such as time frame adjustments. MQL5, however, offers adaptability without the need to manage numerous models for varying conditions.

Author: Gamuchirai Zororo Ndawana

[Antonio Simon Del Vecchio]

Thank you very much for sharing this information and for taking the time to explain it in a very pleasant and readable manner. Excellent.

Regards,

Antonio

[Gamuchirai Zororo Ndawana]

Antonio Simon Del Vecchio #:

Thank you very much for sharing this information and for taking the time to explain it in a very pleasant and readable manner. Excellent.

Regards,

Antonio

You are too kind Antonio, thank you for your kind words.

[Douglas I]

Thank you so much Gamuchirai, this is amazing, well timed and relevant in our EA trading age. 

You're the best :-)

[Gamuchirai Zororo Ndawana]

Douglas I #:
Thank you so much Gamuchirai, this is amazing, well timed and relevant in our EA trading age. 

You're the best :-)

Thank you Douglas, I'm glad to help.

Let's go.🔥

[linfo2]

Thank you for posting , once again a very unique approach. I am excited interested to understand and implement the appraoch. I comes with such and Informative Explanations, It is a pleasure to read and improves my understanding greatly . Appreciate you time spent creating this 

Regards Neil

[Gamuchirai Zororo Ndawana]

linfo2 #:

Thank you for posting, once again a very unique approach. I am excited interested to understand and implement the appraoch. I comes with such and Informative Explanations, It is a pleasure to read and improves my understanding greatly . Appreciate you time spent creating this 

Regards Neil

Thank you, Neil, there are so many people that help behind the scenes like Rashid and other content moderators that make all of this possible, we couldn't possibly thank everyone. I'm glad to have added value, that's always the goal. In fact, it's the only goal.

[Gunnar Forsgren]

Trying the LinearRegressionEA on a weekend I get undefined values for  values m[0] and b[0] in the LinearRegression,mqh, class Predict() metod thus returns undefined -nan, doing back testing in Strategy Tester.
Method Trained() has returned true at that point.
Could it have something to do with being outside trading hours ?

UPDATE: It's likely an issue with historical data. I run in 'real ticks' mode. Trying backtest on one month later of data it works. I'll see if I can add some code that does not let the undefined data slip through undetected. I see there are some checks in there for undefined m[0] and d[0] but may need more of it. I'll check tomorrow.

[Gunnar Forsgren]

Gunnar Forsgren #:
Trying the LinearRegressionEA on a weekend I get undefined values for  values m[0] and b[0] in the LinearRegression,mqh, class Predict() metod thus returns undefined -nan, doing back testing in Strategy Tester.
Method Trained() has returned true at that point.
Could it have something to do with being outside trading hours ?

UPDATE: It's likely an issue with historical data. I run in 'real ticks' mode. Trying backtest on one month later of data it works. I'll see if I can add some code that does not let the undefined data slip through undetected. I see there are some checks in there for undefined m[0] and d[0] but may need more of it. I'll check tomorrow.

My 'every tick' backtest observation got resolved by explicitly exporting Ticks and Bar history data files from the Symbols list and saving them to a file path where the tester will reference them.
The files need to cover the range of days for which tests are performed. This can be convenient for a use case where a period of the past is repeatedly used for improving on a strategy.
By default the tester downloads this data from the net on every test run and apparently sometimes this is not reliable. Keeping this data locally seems to work better.
Usually first time trying to have the tester reference them locally there is some file path acrobatics until coming to know the proper path,
for illustration in my case the Windows path is "C:\Users\<username>\Appdata\Roaming\MetaQuotes\Terminal\Common\Files".
This post provided hints: https://www.mql5.com/en/forum/367098#comment_21816898

[Gamuchirai Zororo Ndawana]

Gunnar Forsgren #:
Trying the LinearRegressionEA on a weekend I get undefined values for  values m[0] and b[0] in the LinearRegression,mqh, class Predict() metod thus returns undefined -nan, doing back testing in Strategy Tester.
Method Trained() has returned true at that point.
Could it have something to do with being outside trading hours ?

UPDATE: It's likely an issue with historical data. I run in 'real ticks' mode. Trying backtest on one month later of data it works. I'll see if I can add some code that does not let the undefined data slip through undetected. I see there are some checks in there for undefined m[0] and d[0] but may need more of it. I'll check tomorrow.

I love your proactive approach. You're right, there are several exceptions that may get raised when trying to fetch historical data. For example, if you try to change time frames in the middle of a trading session, the "-nan" issue may be observed yet again.

There was an inherent tradeoff between keeping the message easy to follow and fixing all the bugs I observed. If I went for the latter, the code may have been necessarily more complex and not as easy to follow as it is. So I decided to keep it easy to follow, with the intentions that you would be able to rapidly extend it.

Your solution sounds very promising, how is it turning out?

[Gamuchirai Zororo Ndawana]

Gunnar Forsgren #:

My 'every tick' backtest observation got resolved by explicitly exporting Ticks and Bar history data files from the Symbols list and saving them to a file path where the tester will reference them.
The files need to cover the range of days for which tests are performed. This can be convenient for a use case where a period of the past is repeatedly used for improving on a strategy.
By default the tester downloads this data from the net on every test run and apparently sometimes this is not reliable. Keeping this data locally seems to work better.
Usually first time trying to have the tester reference them locally there is some file path acrobatics until coming to know the proper path,
for illustration in my case the Windows path is "C:\Users\<username>\Appdata\Roaming\MetaQuotes\Terminal\Common\Files".
This post provided hints: https://www.mql5.com/en/forum/367098#comment_21816898

Yes indeed, having the data locally is a major improvement.  By having the data locally we set the table for going beyond just modifying the coefficients of our linear regression, we would also be able to optimize the parameters of our technical indicators on the fly.  You hit the nail on the head.

Thank you for sharing the link. I'll find time to go through it.