Machine learning in trading: theory, models, practice and algo-trading - page 240

 

coached kohonen on prices

O,H, L, C,

O[-1], H[-1], L[-1], C[-1]

all calculations were performed relative to the current opening as in the column names

> head(dat)
       H/O       L/O       C/O      O1/O     H1/O      L1/O      C1/O
1 1.004326 0.9986890 1.0011799 0.0000000 0.000000 0.0000000 0.0000000
2 1.000000 0.9962027 0.9968574 0.9988215 1.003143 0.9975121 1.0000000
3 1.005518 0.9989490 1.0045980 1.0032843 1.003284 0.9994745 1.0001314
4 1.000392 0.9966000 0.9975154 0.9954230 1.000915 0.9943769 1.0000000
5 1.006949 1.0000000 1.0038023 1.0026223 1.003016 0.9992133 1.0001311
6 1.005877 0.9993470 1.0045710 0.9960820 1.003004 0.9960820 0.9998694

that's how it was done in those links I gave

kohonen divided the data into 100 clusters, it's quite a lot, on those sites they divided it into 5-6 clusters, the accuracy of candlestick patterns should have been a few orders of magnitude higher...

But in fact the quality of recognition is terrible, even recognition is not called

м

=====================================

I got better clustering with the nearest neighbor's method (kmeans) but the results are still not satisfactory

then decided to visualize the cluster

Ideally it should be like this

ь

but with 50 clusters it was like this

о

===========================================

Conclusion

So before you talk about noise it is necessary to transform data so that MO could understand this data, maybe that is why MO was trained better on random because random is not exactly random, when it is generated it is subject to some strict limitations on deviation, variance, etc. So it is more stationary, am I rightvizard?

 
Vizard_:

Well the guys just figured out, by simple examples, that it is easier to get into fewer clusters)))
=============================================
Yeah, that's why I suggested watching the woodsman build trees...

What is the point of having fewer clusters if this thing with a hundred clusters sometimes confuses the color of the candle, not to mention any candlestick combinations

I don't remember anything about wood...

 
mytarmailS:

I don't get it, though.

How was the target one made?

Where did the formula come from?

Yes, I do not know it myself. So he's a magician :)

The point is that he told me which candlestick characteristics to use to recognize patterns, all the rest is a nuanced realization.
I can't create formulas, but for example I want to do the following - to cluster these predictors (candlestick characteristics), evaluate profitability of trade by each individual cluster, divide them into three groups buy/sell/exit according to the average price movement after each cluster. And then something like a forest can get logical rules instead of clustering, but it is not really needed even, any new data can be clustered by previously obtained rules according to the model, and by the number of the cluster to make a decision.

 
Dr.Trader:

I want to do the following - to cluster these predictors (candlestick characteristics), evaluate profitability of trading by each individual cluster, divide them into three groups buy/sell/exit according to the average price movement after each cluster. And then something like a forest can get logical rules instead of clustering, but it is not really needed even, any new data can be clustered by previously obtained rules according to the model, and by the number of cluster to make a decision.

Well I did something very similar, I don't know what kind of target you're going to map to clusters

I took spreads... I broke it down into 100 to 200 clusters.

At quotes there were 2-10 +- interesting clusters that earned something, at oos all plummeted

On randome found 2-7 +- interesting clusters, on oos (on real quotes) somewhere 30-60% of clusters earned, some very stable

But the first place here is the problem of proper data pre-processing, when I visually analyzed what is in those clusters then I got upset, for example if we have a cluster of two candlesticks, then one cluster may well have two white and two black candles, that is two diametrically opposite situations in one cluster, you know how bad it is, so I need a decent data pre-processing so the MO is not so dumb, because the good from such clusters is the same as flipping a coin

 

Take a look at this article, I think it will be useful.

Good luck

Порождение и выбор моделей машинного обучения. Лекция в Яндексе
Порождение и выбор моделей машинного обучения. Лекция в Яндексе
  • habrahabr.ru
Применение машинного обучения может включать работу с данными, тонкую настройку уже обученного алгоритма и т. д. Но масштабная математическая подготовка нужна и на более раннем этапе: когда вы только выбираете модель для дальнейшего использования. Можно выбирать...
 

There is a persistent smell of shamanism from all these exercises with candles. There is no regular thought at all! Just a striking composite of high-brow machine learning algorithms and typical technical analysis nonsense.

For some reason there are no efforts to use other currency pairs as predictors at all. There are plenty of related currency pairs. The simplest one: for a currency pair, which is a target one, we select predictors derived from this currency pair. Then we take the same predictors from other currency pairs.

We obtain a heap of predictors and then let's check the entire heap of obtained predictors for their influence on the target variable. This step is mandatory. Why run garbage around the room with a broom?

And then everything else.

Note that in my proposal there is some idea of "linking currency pairs to each other."

If we are trying to generate some set of predictors, there must be an idea. CAN'T BE THIS IDEA: WE TAKE TWO CANDLES, THEN THEIR COLOR, THEN TWO LAPS AND FINALLY THREE LAPS.

 
SanSanych Fomenko:

There is a persistent smell of shamanism from all these exercises with candles. There is no regular thought at all! Just a striking composite of highbrow machine learning algorithms and typical technical analysis nonsense.

For some reason there are no efforts to use other currency pairs as predictors at all. There are plenty of related currency pairs. The simplest one: for a currency pair, which is a target one, we select predictors derived from this currency pair. Then we take the same predictors from other currency pairs.

We obtain a heap of predictors and then let's check the entire heap of obtained predictors for their influence on the target variable. This step is mandatory. Why run garbage around the room with a broom?

And then everything else.

Note that in my proposal there is some idea of "linking currency pairs to each other."

If we are trying to generate some set of predictors, there must be an idea. CAN'T BE THAT IDEA: TAKE TWO CANDLES, THEN THEIR COLOR, THEN TWO LAPS AND FINALLY THREE LAPS.

=======================================

Are you talking about the material in the article or in general?

 
Vizard_:
When it seems, it is necessary to be baptized. Now ssa)))

"For example, the activation functions that are often used in networks. They can be used not only at the output of the networks, but also at the input" = a prequel
of a book (the book is not bad, I was leafing through it long ago) of twenty years ago. Neurocomputing and its applications in economics and business.
(in Russian), p. 130. Individual Data Normalization.
http://www.neuroproject.ru/Papers/EzSh/Lecture_7.pdf

"An important and well working technique is to use the parameters of the "Singular structural analysis" or "Caterpillar" method." 1996г.)))
http://www.gistatgroup.com/gus/ex1.html

You're too young to give me advice.

And the S.A. is a wonderful tool in capable hands.

Good luck

 
SanSanych Fomenko:

There is a persistent smell of shamanism from all these exercises with candles. There is no regular thought at all! Just a striking composite of high-brow machine learning algorithms and typical crap a la technical analysis.

Sanych well take it and do it!!!

1) Tell the gist of the idea.

2) write and post the code

3) show the pictures of trade on the oos

just blah, blah, blah...

Make at least one normal post, which can not in words but to specific studies to confirm your point of view, which you so fiercely promote, and that it was possible to touch, but you do not do it, right? I know why ...

 
Dr.Trader:

I am not able to create formulas, but for example I want to do the following - to cluster these predictors (characteristics of candlesticks), assess profitability of trade by each individual cluster, divide them into three groups buy/sell/exit according to the average price movement after each cluster. And then something like a forest can get logical rules instead of clustering, but it is not really needed, you can cluster any new data according to previously obtained rules according to the model and make a decision based on the number of the cluster.

Tried it, didn't work. It is possible to pick up clusters for quite nice trade on sample, but on oos almost always plum, bad strategy.
Reason: