Machine learning for robots

[Ivan Negreshniy]

Hi all, I am doing machine learning (ML) of EAs and indicators and have decided to bring my experiments to the public for discussion.

Experiments include generating initial tasks - templates, machine learning, creating robots - EAs or indicators, and testing them.

To try it out, I have already performed all the described steps in the corresponding thread of the MO forum and even invited participants, but unfortunately it did not resonate there, apparently theoretical, scientific discussions of the topic are more interesting there. https://www.mql5.com/ru/forum/86386/page1035#comment_8231641

Therefore, I decided to create a new thread in the robots section and try again, this time inviting everyone, including those not related to DOD, to participate in the experiment.
Once again, no special skills or knowledge in the field of MO or programming is required to participate in the experiment, only autotrading.

Now on to organising and discussing experiments:

• Any willing author creates templates with their strategy's trading signals and posts them in this thread.
• I process templates, create Expert Advisors or indicators and place them here compiled.
• Everyone else is free to download templates and robots, test them and give their expert opinion.

To adhere to the authorship right, all trading robots will be locked to use only on MetaQuotes-Demo. You may also restrict their usage on trading instruments specified by the authors.
Ratings can be set on results of testing and only authors can be given source codes.

In an effort to motivate you to participate in the experiment, I am showing screenshots of the forward testing of the EA_EURUSD_H1_RF placed at the specified link.

This is the result of the entire forward section at the moment where no training has been performed.

Anyone can check this result on MetaQuotes-Demo and monitor further trading dynamics in the future.

[Alexey Volchanskiy]

Ivan Negreshniy:

Now for organising and discussing experiments:

• Any willing author creates templates with trading signals of his strategy and posts them in this thread.
• I process the templates, create Expert Advisors or indicators, and post them here in compiled form.
• Everyone else is free to download templates and robots, test them and give their expert opinion.

To respect the copyright all the robots will be locked to use only on MetaQuotes-Demo, if desired, you can also restrict its use on trading instruments specified by the authors.
The ratings can be set on the results of testing and only the authors can be given the source code personally

I want to disappoint you, you will not be allowed to download them in compiled form. Yes, there is no such prohibition in the forum rules, but this thread will be taken down. I recently had a similar experiment taken down in the English part. Check my personal message, I'll write there recommendations where it's possible.

[Ivan Negreshniy]

Alexey Volchanskiy:

I want to disappoint you, in compiled form you won't be allowed here. Yes, there is no such ban in the forum rules, but the thread will be taken down. I recently had a similar experiment taken down in the English part. Check my personal account and I'll post recommendations on where to do it.

Thanks, but I have a different case - the source code is automatically generated by the author's template and I can, in theory, only give it out to the author, who will decide to publish it himself.

[Ivan Negreshniy]

The activity of the experiment participants is not yet off the scale, but the test EA continues to show 100% profitability:)

...already 11 trades in profit, this with a training sample of less than a thousand signals and training time of just a couple of seconds.

[Ivan Negreshniy]

I suggest using the makeSignals Expert Advisor to automate pattern preparation. The Expert Advisor plots trading signals in the form of arrows on the chart.

Once the signals have been applied a trader can evaluate them, correct them by moving, removing or adding new ones, and then save it all to the template file (menu - Charts/Template/Save Template...).

The Expert Advisor has the following settings:

• Signal bars count - number of bars on which the signal is calculated
• Buy signal pips - estimated number of points of profit for the buy signal
• Sell signal pips - calculated number of points of profit for the sell signal
• Start date time - beginning of a period in which signals are calculated and applied
• End date time - end date of the period in which signals are calculated and applied
• Arrow drawind type - type of the graphical object - arrows used for drawing signals
• Indicator used type - type of indicator used as a filter of signals
• Clear all on exit - delete all graphical objects when disconnecting the Expert Advisor

The Expert Advisor searches within a given interval and plots on the chart all the signals that match the calculated parameters (number of bars and number of pips) and can also filter them if you select the indicator used so far only two are available - ZigZag indicator and crossover of slow and fast EMA.

Information about the signals is displayed in the comment line - they are interval, size in points and the current number of BUY and SELL signals, respectively.

[Aleksey Vyazmikin]

Ivan Negreshniy:

Now for organising and discussing experiments:

• Any willing author creates templates with trading signals of his strategy and posts them in this thread.
• I process the templates, create Expert Advisors or indicators, and post them here in compiled form.
• Everyone else is free to download templates and robots, test them and give their expert opinion.

It is not clear what a template should look like - a naked chart with up and down arrows? Or is it required to attach the indicators on the basis of which the signals were generated?

[Ivan Negreshniy]

Aleksey Vyazmikin:

It is not clear what the template should be - a bare chart with up and down arrows? Or is it required to attach the indicators from which the signals were generated?

The template may contain any indicators, graphic layout, etc., but when generating the robot, only signals will be used, i.e. several arrow types (see Arrow drawing type - EA makeSignals).

Perhaps in the process of experimentation we will formalize some other elements of trading strategies, so they can also be involved and used in templates and code generation.

PS: There are also special templates that have slightly different formats and signals, such as visual testing charts of EAs, trading history and others, with such templates we will have to conduct experiments a bit differently, but I will write about that separately.

[Aleksey Vyazmikin]

Ivan Negreshniy:

The template may contain any indicators, graphic layout, etc., but only signals, i.e. several arrow types will be used when generating the robot (see Arrow drawing type - EA makeSignals).

Perhaps in the process of experimentation we will formalize some other elements of trading strategies, so they can also be used in templates and code generation.

Tell us about the process of processing these arrows. It is not quite clear how algorithms are created based on them, what features are used to bind these arrows, how patterns are formed? If I understand correctly, the goal is to put arrows outside the template markup as well.

[Ivan Negreshniy]

Aleksey Vyazmikin:

Tell us about the processing of these arrows. It is not quite clear how algorithms are created based on them, what features are used to bind these arrows, how patterns are formed? If I understand correctly, the goal is to put arrows outside the template markup as well.

I deliberately keep it simple so as not to overload or scare anyone with all the subtleties of machine learning - working with predictors, models, etc., because in fact, only target signals are absolutely necessary input data, and the rest can be selected and iterated differently, and ideally automated. The main thing is for the author of the template to concentrate on the proper arrangement of signals, according to his experience and strategy, taking into account trading cycles, sessions, news, etc., to make exceptions and correct indicators, etc.

I can tell you, as someone who is familiar with the MO, that at this stage of algorithm development we use patterns with several variants of formulas for calculation of bar and volume price attributes and several variantsof regression models, which are serializable and convertible to MQL, as predictors. In short, make your own author's template, I'll generate it for you, and there, in the source code, as a programmer, you'll see for yourself.

[Aleksey Vyazmikin]

Ivan Negreshniy:

I deliberately choose not to complicate things so as not to overload or frighten anyone with all sorts of subtleties of machine learning - working with predictors, models, etc. In fact, the absolutely necessary input data are only target signals, and the rest can be variously selected, iterated, and ideally automated. The main thing is for the author of the template to concentrate on the proper arrangement of signals, according to his experience and strategy, taking into account trading cycles, sessions, news, etc., to make exceptions and correct indicators, etc.

As someone who is familiar with the MO, I can tell you that at this stage of algorithm development we use patterns with several variants of formulas for calculation of bar and volume price attributes and several variants of regression models, which are serialized and converted to MQL. In short, make your own author's template, I'll generate it for you, and there, in the source code, as a programmer, you'll see for yourself.

If you have automated the process you are talking about, then maybe you should pay attention to the signals service (for simplicity without martin), and try to select targets there?

Well, the arrows to put tedious, I can drop a template after testing with arrows from the trades, will it go?

However, I want to clarify that there on the control of the position? I just do not have one with fixed take and stop - my exit is always by trawl.

[Ivan Negreshniy]

Aleksey Vyazmikin:

If you have automated the process you are talking about, then maybe you should pay attention to the signals service (for simplicity without martin), and try to select targets there?

Well, the arrows to put tedious, I can drop a template after testing with arrows from the trades, will it go?

However, I want to clarify that there on the control of the position? I just do not have one with fixed take and stop - my exit is always by trawl.

As for signals, there the template can be obtained directly from the terminal panel, by <Show on chart> button, and by the existing EA, after testing with visualization enabled and both these special templates are quite suitable for code generation, but as I mentioned above, this is a slightly different topic.

Since it would be better to generate filters for improving initial EAs rather than new ones on the basis of such templates, I don't want to bring up this issue yet because it would cause unnecessary confusion.

I think we should start experiments with examples of the simplest Expert Advisors.

You can use makeSignals for automation of arrows - https://www.mql5.com/ru/forum/270216#comment_8277261

As regards the trawl, we can use the settings of generated EAs, see EA_EURUSD_H1_RF as an example.