Machine learning in trading: theory, models, practice and algo-trading - page 1517
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Question 01.
The following indicators have only one external input parameter
I have chosen these indicators because of the minimum of external input parameters.
Is it possible to reproduce their algorithms with the help of MO methods. I.e. take any history, run the indicators with any parameters and feed them to MO. Is it possible to get the appropriate indicator algorithm as an output?
The question is about the definition of the problem for the MO, i.e. we should mark up the history, I think PriceChannel is easy to teach, ZZ may be second in complexity but Exponential Smoothing is not a classification, it's regression and probably the same thing, but I haven't worked with regression.
Question 03.
I have encountered such situations in practice several times. The TS shows profit on the testing interval. When I run OOS for several times more, it shows the same profit. I have tested it on real account and it shows the same profit for several months.
And at a certain moment it systematically sinks. No reoptimization of the same TS gives no positive results. In the best case it slows the plummet.
After a while the programmer understands that everything sucks and leaves the real account. You watch for a few months how the TS is losing in the tester.
And suddenly, two weeks of profit, a month of profit, OOS-month - profit. This is still the same TS. You put it on the real and get everything the same as described in the first paragraph.
This is not a hypothetical situation, but a case of practice. In this case the TS demonstrated its graality only on a small number of symbols. On others it was a year-round drain.
And, of course, the TS did not have to trade around the clock. There could have been an intraday trading interval.
In terms of common MO application practices, was it possible to get a TS with similar characteristics?
Question 02.
Let the TS with four input parameters perform thousands of transactions on some interval. Add as a filter two input parameters with a low number of possible variants. At the output we have a straight up graph with about one thousand trades. And all of them are more or less evenly distributed over the entire testing area.
What is the reason of the high probability of losing 5% of the initial interval on OOS? A huge interval and only six inputs gave an upward straight line on a really large number of trades. And what came out was a bullshit fit.
Does this mean that there are actually more than six input parameters? It's kind of a reference to the first question- aren't the algorithms that are simple to us really complex by nature?
And why shouldn't there be a plum? What is the similarity in historical data is a separate question to assess the correctness of the model.
And then what was given as input, how the history was marked out for training, what was taught - there are a lot of nuances here.
Question 03.
I have encountered such situations in practice several times. The TS shows profit on the testing interval. When I run OOS for several times more, it shows the same profit. I have tested it on real account and it shows the same profit for several months.
And at a certain moment it systematically sinks. No reoptimization of the same TS gives no positive results. In the best case it slows the plummet.
After a while the programmer understands that everything sucks and leaves the real account. You watch for a few months how the TS is losing in the tester.
And suddenly, two weeks of profit, a month of profit, OOS-month - profit. This is still the same TS. You put it on the real and get everything the same as described in the first paragraph.
This is not a hypothetical situation, but a case of practice. In this case the TS demonstrated its graality only on a small number of symbols. On others it was a year-round drain.
And, of course, the TS did not have to trade around the clock. There could have been an intraday trading interval.
In terms of common MO application practices, was it possible to obtain the TS with such characteristics?
So this is again a question related to market phases and what the TS/model MO learned. For example I trained the model on the CatBoost - all good on the history outside of training, put it on the real and the silence of the whole 3 months, the model was silent - withdrew, and now for the last month through the tests I see that I missed a lot of profitable deals (last month), but then apparently the model itself has something considered and not trade, because the market really were very flat period, and the model is set to trend (on the minutes).
Why shouldn't there be a flush? What is the similarity of historical data - this is a separate question to assess the correctness of the model.
And then, what was given for input, how the history was marked for training, what was taught - there are a lot of nuances.
What training? A simple classic Expert Advisor with six optimizable parameters. The regular GA shows excellent results on the interval.
I'm not interested in the answer "of course I lost on OOS". And for what reason is the reason for the loss on the OOS, even with a thousand trades?
My amateur reasoning is as follows.
There are very few degrees of freedom in the TS. So the probability of adjustment decreases.
Thousands of trades evenly distributed on the interval - this means that the long-lasting "pattern" on this interval has been detected. Especially since the balance graph is almost a straight line.
What is actually wrong in this reasoning? I assume that in fact the input parameters are not six, but conventionally a million. This is the number of input parameters of the universal machine, which reproduce the TC algorithm. That is, the algorithm itself is a certain set of input parameters.
What training?! The usual classic Expert Advisor with six optimizable parameters. Regular GA gives excellent results on the interval.
I'm not interested in the answer "what were you expecting, of course I was losing on OOS". And for what reason is the reason to lose on OOS, even with a thousand trades?
And what is the learning interval? 1000 trades can be on one global trend, where opening positions on its vector will give an advantage in itself.
For example, I train Si futures on the history of 5 years, so that there were more different situations. From 2008 to 2016 I optimized Forex Expert Advisors on 15 minutes - they are viable enough for the averaging.
And in fact the optimization is the same MO method, so all of the above is true.
My amateur reasoning is as follows.
There are very few degrees of freedom in TC. So the probability of adjustment decreases.
Thousands of trades evenly distributed on the interval - this means that the long-playing "pattern" on this interval has been detected. Especially since the balance graph is almost a straight line.
Why do not you want to think about assessing the similarity of the markets - for example I see that on forex volatility has fallen markedly over the past few years, even this year - I used to earn 3-4 times more per month, and now there is just no movement and no trades on my TS on forex.
And then, everything has to be looked at objectively, try to understand what the TS does, if it's really simple and think if its actions are reasonable or if it's a matter of chance.
What is really wrong with this reasoning?
Erroneous reasoning, for it assumes the stationarity of the market, which is not the same...
I assume that in fact the input parameters are not six, but conventionally a million. This is the number of input parameters of the universal machine, which reproduce the TS algorithm. In other words, the algorithm itself is a certain set of input parameters.
If we are talking about the indicators, there are much more parameters, because we should consider the logic of the indicator itself, almost every if in the indicator code as a separate condition, which of course will produce a countless number of combinations. However, I personally think that on the contrary, the use of indicators is a common sense approach, because I assume that many market participants use them, which means there is a probability to catch a piece of the logic of a participant's TS with money.
Question 01.
The following indicators have only one external input parameter
I have chosen these indicators because of the minimum of external input parameters.
Is it possible to reproduce their algorithms using MO methods? I.e. take any history, run the indicators with any parameters and feed them to MO. Is it possible to obtain the corresponding indicator's algorithm in the output?
EMA-like smoothing can be obtained exactly, I hope you mean the result, not the "algorithm" in the form of code :)
I'm not so sure about PriceChannel and ZZ, they will definitely be flawed. In general, the task is interesting, thank you:)
Question 02.
Suppose that on some interval the TS with four input parameters makes thousands of deals. Add as a filter two input parameters with a low number of possible variants. At the output we have a straight line chart with approximately one thousand trades. And all of them are more or less evenly distributed over the entire testing area.
What is the reason of the high probability of losing 5% of the initial interval on OOS? A huge interval and only six inputs gave an upward straight line on a really large number of trades. And what came out was a bullshit fit.
Does this mean that there are actually more than six input parameters? It is sort of a reference to the first question- aren't the algorithms that are simple to us really complex by nature?
This is more a question of type of strategy algorithm and MM than of prediction. Most of the tester grails by type are reversible with averaging and sometimes oversaturation, on such sets it is very easy to get a "grail" with one parameter, in order to have a more objective picture, you should at least remove averaging, though it is not enough, IMHO the correct way should measure expectation of prediction (trends, reversals), because you can cheat well not only others but yourself too.
Question 03.
I faced such situations in practice several times. The TS shows profit on the testing interval. When I trade with OOS for several times more, the same profit is shown. Bet on a real account and it shows the same profit for several months.
And at a certain moment it systematically sinks. No reoptimization of the same TS gives no positive results. In the best case it slows the plummet.
After a while the programmer understands that everything sucks and leaves the real account. You watch for a few months how the TS is losing in the tester.
And suddenly, two weeks of profit, a month of profit, OOS-month - profit. This is still the same TS. You put it on the real and get everything the same as described in the first paragraph.
This is not a hypothetical situation, but a case of practice. In this case the TS demonstrated its graality only on a small number of symbols. On others it was a year-round drain.
And, of course, the TS did not have to trade around the clock. There could have been an intraday trading interval.
In terms of common MO application practices, was it possible to obtain the TS with such characteristics?
Hardly anyone can comment such characteristics "with his hand on the heart". In general you shouldn't consider just MoM as a separate thing, MoM is just an extension of statistics, a general algorithm is MoM, even mesh optimization is MoM, so there has always been MoM in algorithmic trading:)
There is a tendency for lazy people to hope for full automation of searching for TC and let IR do everything for them, fortunately it won't happen for a long time, and when it does it won't have financial sense, and meanwhile classic IR is bad looking for sensible signs, and "deep" is very capricious in all respects, it is very painful to work with it and the result is nebulous.
My amateur reasoning is as follows.
There are very few degrees of freedom in the TC. This means that the probability of adjustment decreases.
Thousands of trades distributed evenly on the interval - it means that the long-lasting "regularity" on this interval has been revealed. Especially since the balance graph is almost a straight line.
What is actually wrong in this reasoning? I assume that in fact the input parameters are not six, but conventionally a million. This is the number of input parameters of the universal machine, which reproduce the TC algorithm. That is, the algorithm itself is a certain set of input parameters.
It is erroneous, that when estimating a time series (no matter if it is a price, a sequence of falling dice or numbers of "the wheel of Fortune"... or even a temperature chart)
You (and we all participants of this forum) assume that the time series must behave in the future exactly as we tried! Unfortunately, the fit is always present, and the more we use input data, the greater the fit and our confidence
read the first part of the article on hubrahttps://habr.com/ru/company/ods/blog/322716/
before the graph
really liked the presentation of the material
Well and about your question with MO, my dilettante reasoning:
the problem to prepare input data for training, with ZigZag in general a trouble - the simplest feed prices fracture ZZ - trained, tested - does not work, why? - because it is tightly bound to specific data, in the future there are no such prices, NS is a regular polynomial y = ax+bx+cx+dx+ex..... the number of polynomials is the number of neurons, the more neurons, the better quality (NS error), but faster re-training occurs.... overlearning is fought by inventing new types of NS, but there, too, where it is better where it is worse....
but with periodic functions NS will work, in fact, perfectly - a graph of sine/cosine at any scale and with minimal data for learning - because such functions can be written using Taylor series expansion? (I do not remember already) - and it will be five polynomials y = ax+bx+cx+dx+ex
SZY: with price series no one managed to invent polynomial formula - that's why NS doesn't work perfectly... Maybe this formula does not exist )))
This is more a question of the type of strategy algorithm and MM than of prediction. Most of the tester grails by type reverse with averaging and sometimes oversaturation, on such sets it is very easy to get a "grail" and with one parameter, in order to have a more objective picture, you should at least remove the averaging, although it is not enough, IMHO the correct way to measure expectation of prediction (trends / reversals), because the tester can easily fool not only others but also yourself loved one.
I was spinning something tonight. As an unquoted answer, I think it will be interesting for others to read.
https://www.mql5.com/ru/blogs/post/728196
1. Yes, with some accuracy (approximation error)
2. Take a 3rd degree polynomial (only 3 free terms) and fit it to a kilometer long piece of graph. Does it mean that it will work on the OOS? Of course not. The curves will start to diverge almost immediately, but sometimes you can get lucky with the direction. If the pattern is unknown (theoretically, conceptually, fundamentally - whatever) that is being exploited, then the question can be dismissed, as it is always a fit for SB
3. in terms of MO you can get anything you want. Basically, these sections will be with certain trends, if they are similar then it will work. I.e. in this case it is the global trend that changes, most likely, or again the market becomes similar to what it was, in the medium term. That's why it works/doesn't work.
All these issues are solved in econometrics at the conceptual level: the linear trend is the main component, it is forecasted first. Then nonlinear fluctuations around a straight stick, provided that the trend persists. The most nagloblest trend cannot be predicted due to lack of itoria, i.e. all uncertainty will be from the long-run plan. Just a few free terms, usually not more than 3, describe any market curve. The MOS can already be adjusted to the residuals of what is left unpredicted.