Machine learning in trading: theory, models, practice and algo-trading - page 1322
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Guys, the topic is about MO, not about trading styles and stops. What does it have to do with them?
I wanted to discuss it in NS, but as it turned out there is no one to discuss it with.
As recently as yesterday, the conversation was about predicting sinusoids, and I remembered my old topic:
I am vindictive and I write everything down).
I must say, you were wrongly poked, but the topic got stalled and never got down to business. Perhaps because of the not very. clear wording.
Actually, we don't need to solve any problem. We take a function, such as the one in the thread, or even better, a more complicated one. Create an artificial tool with this function and run it in the tester on an already working strategy. By idea, the profit should go off the charts on the working TS. Oh, I forgot, the function should be normalized in advance so that it approximately corresponds to the symbol to which the TS is tuned. Then we can add some noise and see what will happen.
I don't make forecasts and I don't have such TS ready, so I cannot check it in the nearest future. But in the distant future, I plan to.
Now about why we need all this.
Suppose we need to teach NS (or other MO) forecasting. Usually initial weights of NS are initialized randomly and if the NS gets into min-maxes when training, it's a big question.
Let's do the following.
1. We generate a non-random function close to market BP and use it for training of randomly initialized NS. Check it and so on. Now our NS is close to what we need in terms of settings, but so far it has not been able to solve the real problem.
2. Conduct training of the NS (see point 1) using real BP. At the same time we already have some guarantees that preliminary NS settings are already somewhere in the vicinity of min-max areas, and during additional training they will go where they should, but not to some random min-max.
The analogy is with a high school student who is first taught to solve simple problems on a topic, and then those problems are made more difficult. Teaching à la schoolboy is more effective than trying to make you solve complex problems at once.
In general, the method is not an opening, somewhere in the literature it was found, but there are many books, and I am alone - I do not remember. In any case, I thought about its implementation. Well, and the very first experiment with the attempt of the ready TC to predict the analytical function, in general, is necessary, as staged.
if this one hadn't flubbed, 2 pages ago you would have seen an example with the f-x
I don't get the idea about initializing weights and stuff, then you need a pre-training grid, and what's the point of training on something that doesn't match what you need to predict at all
I guess you are trying to represent the basics of learning with reinforcement
If the bastard hadn't flubbed, 2 pages ago you would have seen an example with a f-eye
I don't get the idea about initializing weights and stuff, then you need a pre-learning grid, and what's the point of training on something that doesn't match what you need to predict at all
I'm guessing you're trying to depict the basics of reinforcement learning
No, you don't need a pre-learning grid.
1. Pre-training on near and clearly predictable data. You can prepare from real data - splines, polynomials, Fourier, or whatever.
2. Post-training of this network according to p. 1 (with non-random initialization) on the real data.
For forecasting, I can see how to do it, and I think it may improve the results. For classification, I have no idea.
The example with the function is not about that. There are a lot of such examples.
Alexey chose not the worst catbust, one of the best for research, today. It is used at CERN, for example, to analyze collider results... who deals with quantum randomness.)
Very interesting info) CERN XGBoost came across by accident.
Not surprisingly, CERN recognized this as the best approach for classifying Large Hadron Collider signals. This particular problem posed by CERN required a solution that was scalable to handle data generated at a rate of 3 petabytes per year and effectively distinguish an extremely rare signal from background noise in a complex physical process. XGBoost was the most useful, simple and reliable solution. (SEPTEMBER 6, 2018)
__________________________________
CatBoost was introduced by the European Center for Nuclear Research (CERN) in research at the Large Hadron Collider (LHCb) to combine information from different parts of the LHCb detector into the most accurate, aggregated particle knowledge possible. By using CatBoost to combine the data, scientists were able to improve the quality characteristics of the final solution, where CatBoost results were better than those obtained using other methods[6][7]._______________________________________
It's all pretty much the same thing. So which one is Catbust orXGBoost? ) I wonder who cheated from whom? ))
The future is uncertain, patterns appear and disappear, that's normal, but the fact that they necessarily have to be short-term is questionable. My sample is not very large because of the trend strategy, so I think it is unreasonable to reduce it even further.
However, I decided to conduct an experiment on the effectiveness of training on different proportions of the training and test sample involved in the training. Step will be 10%, that is, a training in the beginning of 90% and 10% of the test, then the test gradually increased by 10%, in each case will be 200 models - see what happens. Another question is how best to compare, these combinations, on the average or the absolute criterion - ideas are accepted.
On short-term patterns - from 10-15 min to max 1 hour the probability of any significant events that change something is very small, especially when we have already entered a deal on a significant event. For more than an hour - what will happen there, at least to me, I do not know at all. This is not in opposition to you, but in support of my opinion).
Decided to follow your way, with my modifications and vision of the subject. Especially since I already have all the necessary predictors. So far I've decided to try it with XGBoost, I'm not impressed by CatBoost docs (XGBoost, imho, is clearer) or something else. So far, I haven't understood what's up with rationing. With NS, it's hard to ration everything there.
All are almost identical. So which one is Catbust orXGBoost? ) I wonder who cheated from whom? ))
And it would be good to understand this before to create models based on them and release them into the market chaos.
I remember about half a year ago they tried to test CatBoost, then unlike XGBoost it could not learn even from multiplication table.
I don't know, maybe now they decided that if it can't do simple then they should try it on complex, or maybe it is really cool.
And to determine this, you could probably use the synthetic BPs suggested above.
And it would be good to understand this before creating models based on them and release them into the market chaos.
I remember about half a year ago trying to test CatBoost, then in contrast to XGBoost he could not learn even on the multiplication table.
I don't know, maybe now they decided that if it can't do simple then they should try it on complex, or maybe it's really cool.
And that would determine it probably could be used, offered above, synthetic BP.
At least XGBoost has a lot better documentation, and more. The CatBoost, apart from Yandex's own materials, has very few others, at a glance.
According to short-term patterns - from 10-15 min to max 1 hour the probability of any significant events that change something is very small, especially since we have already entered a deal on a significant event. For more than an hour - what will happen there, at least to me, I do not know at all. This is not in opposition to you, but in support of my opinion).
I don't look at probability variability that way as part of my strategy. If the model gave a signal to enter, then we enter, because right now (simply put by the statistics obtained in the training) there were favorable conditions for it and exit in my different versions of TS is different - either by TP / SL or only by SL. As a result the prediction is not cancelled until the position is closed and may be valid for three hours. That is, I do not forecast exactly the time, but rather the probability of getting back to the price x bars ago, in other words, the end of the local flat.
But I wrote about something completely different, namely about the recurrence periodicity of the identified pattern in time, that if the pattern has occurred during three years and led to predictable events, then there is a chance to a greater extent, that it will continue its work in the fourth year, while the pattern identified in a small portion of the training time may only be a description of the situation in the upper TF (trend on a weekly chart).
Anyway, this is my theoretical reasoning, now I will process the results of the experiment and the situation will become more clear, I hope.
I decided to follow your way, with my own modifications and vision of the subject. Especially, that all necessary predictors already exist. So far I've decided to try XGBoost, something about it struck me - whether the CatBoost docs didn't impress me (XGBoost, imho, is clearer) or something else. So far, I haven't understood what's up with rationing. With NS, it's hard to ration everything.
Catbust has more settings, it is a continuation of the idea of XGBoost, the main plus and simultaneously minus is trees of fixed dimension, which prevent re-learning if their number is not great.
It is not necessary to normalize anything for pure classification. However, I do preprocessing in the predictors, reducing the range of values, and I decompose the range itself into groups empirically. Perhaps the result would have been better without these transformations - I did not check. On the one hand, the developers say that I don't need to convert anything myself, but on the other hand there are different settings for the conversion algorithm, and in addition you can use your own breakdown from a separate file. This applies to catbust, but I think the logic is the same everywhere.
And it would be good to understand this before creating models based on them and release them into the market chaos.
I remember about half a year ago trying to test CatBoost, then in contrast to XGBoost he could not learn even on the multiplication table.
I don't know, maybe now they decided that if it can't do simple then they should try it on complex, or maybe it is really cool.
And that to determine this probably could have been used, offered above, synthetic BP.
CatBoost and XGBoost have different tree depths, I remember that when adding the number of trees CatBoost was successful.
And as for finding different function with trees, it is better to search them with NS, and feed the results in the form of predictors for classification. In any case, I'm not a supporter of raw price feed, but Maxim succeeded in it, though more frequent retraining is required.
For those who doubt the ability of the NS to describe functions