Machine learning in trading: theory, models, practice and algo-trading - page 522

 
Dr. Trader:

It is very dangerous to distribute classes to a teacher at random, like taking some indicator to create teacher classes and then replacing some of the values with NA as well.

Even if there are good predictors and good teacher classes and the model holds good results on new data, any attempt to tweak class values can completely break the model. Finding indicators for predictors and an indicator for classes that will keep the model profitable on new data as well is a great stroke of luck.

I would recommend to start with two simple classes - the color of the next bar (i.e. buy/sell). Take at least 10000 training examples (history bar), train the model and evaluate the result on the next 10000 bars in history (that were unknown to the model during training). When you manage to find the predictors that will maintain the accuracy of the model on the same level on the old and new data - you can start to select an indicator for the teacher's classes. And it will turn out that just taking the first available indicator the model will not preserve the accuracy on new data. Why some indicators can serve for the teacher and some cannot - I do not know, it is some kind of luck and mystic.

Why is it random? Any indicator - the same zigzag gives buy/sell commands when changing direction. And all intermediate ones are of class NA or "wait". In other words, I don't replace the buy/sell class with NA.

I am experimenting with this indicator https://www.mql5.com/ru/code/903 in TP-SL mode, which I then put in the trading part of the Expert Advisor. If someone knows other interesting indicators for NA - send me the link.

Sampler
Sampler
  • votes: 39
  • 2012.06.01
  • Serj
  • www.mql5.com
Индикатор (i_Sampler.mq5) рассчитывает идеальные входы, предназначен для обучения нейросети. Индикатор имеет два буфера: буфер 0 (зеленая линия на картинке) аналоговый сигнал, рассчитывается как отношение положительного и отрицательного отклонения цены за bars_future баров вперед, нормализованное в диапазон -1, +1; буфер 1 (двухцветная...
 
elibrarius:

Why at random? Any indicator - the same zigzag gives buy/sell commands when the direction changes. And all intermediate ones I refer to the NA or "wait" class. That is, I don't replace the buy/sell class with NA.

I experiment with this indicator https://www.mql5.com/ru/code/903 in TP-SL mode which I then place in the trading part of my Expert Advisor. If somebody knows some other interesting indicators for NS - send me the link.


The disadvantage is that your inputs and outputs of this indicator may correlate poorly, it's better to give to the output the same thing as to the input, imho. But the fact that the stops are taken into account is cool.

I tried it both with and without it - with it it is worse :)

 

I experimented more with softmax. The problem really turned out to be a different number of training examples. When aligning them and trade went both ways.

But NS-regression with linear output still behaves robustly on unleveled data. It seems to me to be a more reliable method.

 
No one has ever commented on this:

Why aren't the training examples used in the articles for the moments when you don't have to do anything? Because doing nothing at the right moments is also important, usually these moments are when the trade will lead to a loss.
Without learning how to pause, NS can start trading and lose the deposit.

Update: I think I understand from the articles... The NS should not learn the moments of changes of the zigzag sign, but its direction. And then the trading module should detect changes of direction and trade at those moments. This is for the ZigZag.
And the output indicator iSampler (above) has three classes initially. The third class (NA-waiting) cannot be simply removed otherwise we will get what I have described, i.e. trading at moments when it is not worth trading.
 
elibrarius:
No one has commented on:Update: I think I understand from the articles... I don't think we should learn NS not by the change of the zigzag sign but by its direction. And then use the trading module to detect changes of direction and trade at those moments. This is for the ZigZag.
And the output indicator iSampler (above) has three classes initially. The third class (NA-waiting) cannot be simply removed otherwise we will get what I have described, i.e. trading at moments when it is not worth trading.

This is called "pure creativity", you can do anything, if you have time and desire. What's more, I'm sure there was no universal approach and won't be, there are basic recommendations only for IR itself, but not for strategies on it

 
Задачи классификации временных рядов с утилитой ML-Assistant
Задачи классификации временных рядов с утилитой ML-Assistant
  • 2017.11.13
  • Aleksey Terentev
  • www.mql5.com
В последнее время машинное обучение (МО) становится все более известным в широких кругах. Конечно не обошло оно стороной и тех, кого интересует заработок спекуляциями на международных рынках различных специализаций. Действительно, эта сфера предоставляет огромное количество данных, которые изучаются уже достаточно давно математиками разного...
 

Thank you, good indicator, will save me time.

Here is an example for R, the forest is trained on the data, the model is saved to a file, and when predicting it is loaded from the file and used.
r script should be saved to documents, indicator settings loaded from ML-Assistant.set file. And tweak the file and folder paths for yourself there.

This code is only suitable to show how to organize communication with R using ML-Assistant. All the rest is just a meaningless template, the process of model training is primitive and will not work for forex, we need more crossvalidation and selection of model parameters, as well as indicators and target must also be selected.

 

I saw the theme in the top and could not resist my 5 kopecks. I saw the topic in the top and couldn't resist my own 5 points.

It is a pity that the situation on the real account is completely different. In general I noticed it is difficult to go better than the TS. As a rule the result is a little worse than in the tester.... And sometimes it's even worse.....

 

A little bit about the approach to data compression and decorrelation: PCA, SVD, Autoencoder

https://habrahabr.ru/post/275273/

http://math-info.hse.ru/f/2015-16/ling-mag-quant/lecture-pca.html

https://habrahabr.ru/post/304214/

Autoencoder is used in dithering, but it can be replaced by PCA, e.g. when you want to use many features, but don't know beforehand which ones are more informative. Alglib has PCA and SVD. Of course, it is not certain that the methods will choose the most informative ones, because they find components with the highest variance, but at least they do a good job with decorrelation.
 
Mihail Marchukajtes:

I saw the theme in the top and could not resist my 5 kopecks. I saw the topic in the top and couldn't resist my own 5 points.

It is a pity that the situation on the real account is completely different. In general I noticed it is difficult to go better than the TS. As a rule the result is a little worse than in the tester.... And sometimes it's even worse.....


This is nothing, it's the usual tweaking, it's been discussed 100 times already. It's elementary and represents no practical use in forex.

Reason: