Machine learning in trading: theory, models, practice and algo-trading - page 139
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
All this sounds academic.
As I remember with more than 100,000 observations, even the initial quotient is almost stationary.
So what?
After all, most likely we work on the terminal in a window of 100-150 observations. In general, what should be the size of the window? And for what purpose?
If for the removal of noise predictors, a large number, maybe up to 10 000. Do you need 100,000 to remove noise predictors?
Actually, the sample size has to be somehow related to some stationary characteristics of the market. In the case under discussion, the noise predictors should be so for fairly large time intervals.
This is the noise removal stage.
The next step is to train the model in order to make sure that the model is not over-trained.
If we manage to find a set of predictors that the model is not retrained, then everything else makes sense. Otherwise any talk is completely empty. Let's not forget that it is much easier to build a knowingly over-trained TS using indicators, than using all these artificial intelligence gadgets, as a result of which we still get an over-trained TS.
So we have a certain set of predictors that does not lead to retraining. Profitability value is not important, the main thing is to have such profitability and not to fluctuate much at different samples.
Now comes the next step.
Let's take a small window = 100-500 observations and use it to study the model, selecting predictors, for instance, by rfe. It will reduce the set of predictors even more. By experience in two times.
Using this reduced set of predictors we make prediction exactly one step ahead. Ideally this prediction should have a confidence interval - this is the risk.
Trade inside the prediction. I.e. we predict the next day, and trade on M5. On the half of the predicted movement we put TP.
On exiting all positions we shift the window, select the predictors by rfe and predict again.
Prediction error is an error at each step, not all in one go.
But the practical value is not this value, but the value of the profit factor / drawdown or something else at least in the tester.
It all sounds academic.
///
I will replace all your conclusions with a single phrase: non-noise predictors will work on an infinitely large sample. So, the noise must be screened out on a sample of as large a size as possible.
If your "non-noise predictors" work on a limited sample of the quotient, then they are local noise.
Okay, it will be interesting to read.
CNN is not a recursive network per se.
Speaking of convolutional networks. This kind of neural network is not designed for solving regression problems. That is why the results of our public experiment were disappointing.
The tasks to be solved by CNN are: classification, localization, detection, segmentation.
So, these are "notes in the margin".
Good luck
Speaking of convolutional networks. This kind of neural network is not designed for solving regression problems. That is why the results of the public experiment were disappointing.
The tasks solved by CNN are: classification, localization, detection, segmentation.
These are just "notes in the margin".
Good luck
Are you sure that classification cannot be represented as regression and vice versa? After all, this block is solved on fully connected layers, and what's the difference there? Do you have a pruf?
And by the way the results are not deplorable. Why did they decide that? Just that the inherent function is quite complex and the network ceases to distinguish it from noise (I mean a full-connected network). Have you worked it out yourself, or are we just demagoguing?
Do you guys have intraday quotes, preferably M5 indices, need three instruments, at least 50 000 candles
1) Euro Stoxx 50 orEuro Stoxx 600
2) Dow Jones or S&P 500 or NASDAQ
3) EUR/USD pair
Do you have any, please let me know for experiments, I will be grateful
Alexey Burnakov:
If your "non-noise predictors" are working on a limited sample interval of a quotient, then they are local noise.
What do you want to prove on the longest possible samples? The efficient markets hypothesis? There are plenty of noobiles out there, including the bankrupts who preached that hypothesis.
I, on the other hand, am talking about a very specific application of tools. And this specificity to me is briefly expressed by the phrase: "up-front half." Within this framework, the optimum prediction horizon is 1 week, to put my heart at ease, 2 months.
Everything else has to be adjusted precisely for the withdrawal of profits. Here and now. What was happening before '87 is not interesting. It is not interesting what happened before '97, and if we go further, the next frontier is 2007. When choosing a time period to identify noise predictors, you have to have some kind of meaningful consideration that the preceding period will be similar to the future. Does everything work the same after Brexit as it did before it? and what happens after the U.S. election? There are political shocks and stock market crashes unrelated to politics, and they divide historical data into chunks.
video -https://www.youtube.com/watch?v=-INzzOXxkhU
In my opinion, here is one of the correct ways of development, and it comprehensively solves both the problem of noise selection and retraining(this is all in theory, of course)
1) We look for recurring situations in the history - ala selection of signs
2) We calculate the statistics of the price reaction on the pattern, say 10 declines and one growth, so it turns out that we have done this
1. We have identified strong patterns with good statistics (selection of qualitative signs)
2. We have understood that the pattern is not true, because it was repeated too often (qualitative feature selection)
3. We essentially crosvalidated the trait, as we calculated all statistics on it.
Re-training has been eliminated, as all unnecessary things have been done, leaving only what has worked well on our history
and all this as if in one bottle :)
p.s. when watching the video watch the indicator, I did not see a single time that he was wrong in the forecast, and this is strongly ...
What do you want to prove with the longest possible samples? The efficient markets hypothesis? There are plenty of noobiles out there, including bankrupts, who have preached this hypothesis.
Actually, I'm showing that on the maximum available sample the dependencies on the selected fiches are reproducible, which is the direct opposite of your statement. The market is inefficient throughout history.
I won't comment on the rest of your points: that's your preference.
video -https://www.youtube.com/watch?v=-INzzOXxkhU
In my opinion, here is one of the correct ways of development, and it comprehensively solves both the problem of noise selection and retraining(this is all in theory, of course)
1) We look for recurring situations in the history - ala selection of signs
2) We calculate the statistics of the price reaction on the pattern, say 10 declines and one growth, so it turns out that we have done this
1. We have identified strong patterns with good statistics (selection of qualitative signs)
2. We have understood that the pattern is not true, because it was repeated too often (qualitative feature selection)
3. We essentially crosvalidated the trait, as we calculated all statistics on it.
Re-training has been eliminated, as all unnecessary things have been done, leaving only what has worked well on our history
and all this as if in one bottle :)
p.s. when watching the video watch the indicator, I did not see a single time that he was wrong in the forecast, and this is strongly ...
all true.
I didn't watch the video, as I am convinced that you can fit anything to make it look pretty, as big a story as you want. The truth is not pretty. Proving that the model is not overtrained is very difficult.
For me it's all about the risk - I try to take at least a little risk. It is possible to create an Expert Advisor successfully trading on a dozen pairs for many years, but why? Profit will probably be a couple of percent a year.....
Well, you haven't even tried it... :)
A small report on my experiments with statistical arbitrage.
I tried to handle positions (additions, averaging), I think you can get much better results.
So to say, the new data "holds the punch" arbitrage is much better than those systems that I did using a machine learning, so to speak, it holds 10 times better ...
I've got more in half a day of experimenting with arbitrage than I did in half a year with the machine learning. It's not so much strange as offensive...
From the minuses there are essentially two:
First, there's a big +/-30% drawdown.
And the second, I still can't think of a way to screw in the machine learning :)