Machine learning in trading: theory, models, practice and algo-trading - page 1237
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Two years ago I wrote here Maximka that NS is a toy like a nuclear bomb. That if ANY other model gives at least satisfactory results, it is not recommended to use NS - they find what does not exist and can not do anything with it.
By trees is a good thing, but it is better to use scaffolding.
Will it handle 100,000 samples with e.g. 100 predictors?
I plan to experiment with xgboost through R, I think it has everything I need. And it seems to be very fast and uses memory in an optimal way.
Can it handle 100,000 samples with e.g. 100 predictors?
) I do not know - this is already to Fa, probably.
I used simple models - played with binary.
I plan to experiment with xgboost through R, I think it has everything I need. It seems to be very fast and uses memory in an optimal way.
Can it handle 100,000 samples with e.g. 100 predictors?
really fast
When i learn how to save matrices in mql in a normal form into a file (that's a pain in the ass)) i'll try it on my data
i have no difference in quality between mqlp and logs... in general there is almost no difference between anything and anything from normal models, but i'm interested in trying boisting myself, maybe it's really 40 times better... but in fact it will be a bit better.
concerning the above book - when I see that the text contains too many unnecessary words and interjections I immediately realize that the author is nuts and close it ))
really fast
When i learn how to save matrices in mql in a normal form into a file (that's a pain in the ass)) i'll try it on my data
i have no difference in quality between mqlp and logs... in general there is almost no difference between anything and anything from normal models, but i'm interested in trying boisting myself, maybe it's really 40 times better... but in fact it will be a bit better.
Regarding the book - when I see that the text is filled with too many unnecessary words and interjections, I immediately realize that the author is nuts and close it ))
With forests, I have not yet understood how they deal with noise (except that it is a vote of many different trees).
Can the single tree itself suppress the noise? (From what I read a year ago I remember that it can completely memorize the sample together with the noise)
NS cannot find important data because of BackProp when there is a lot of noise (and all long bars are almost noisy, it is important, in fact, whether they come from the top or from the bottom).
With forests, I have not yet understood how they deal with noise (except that it is a vote of many different trees).
Can the single tree itself suppress noise? (From what I read a year ago, I recall that it can completely memorize the sample along with the noise)
just like ensembles of neural networks are built on bagging and you get the same thing as a forest
you have to understand that a forest is just a special case of bagging, instead of trees you can put there anything, any weak models. A single tree is an elementary model that can't suppress anything
the forest "sort of" doesn't retrain because of random sampling, but in fact it retrains very easily and unpretentiously
both need to be regularized either through Decay (gradient step), early stopping, or through the r parameter of the forest, or do preprocessing, but all these improvements are usually within 5-10%. On bad data both models will work equally bad
about boosting (extreme, not GBM) they say it doesn't retrain too much, you should look it up, i can't say anything
https://www.analyticsvidhya.com/blog/2016/03/complete-guide-parameter-tuning-xgboost-with-codes-python/
My esteemed Innocent. I`m not afraid of that word, Master, thank you for those moments of joy!
Life shone with new colors (blue, red, gray, green))))
If you could please answer a few questions. Why do you so diligently refer to Vorontsov as Voronov.
And why, when you are telling us about the market without MO, you're trying to sell us some kind of a poncy targeting.
And the main thing - where is the trend-flat?)))
I wrote about Vorontsov and my despair at your mockery, but my posts were deleted, don't be like the Grammar Nazis, my target is not useless but - the basis for the Grail, I have gray (flat) and color (trend) differ with 90% accuracy, blue and red for clarity, I do not use direction, green indicator takes an absolute value.
When I learn how to save matrices in mql in a normal form into a file (that's a pain in the ass)), I'll try it with my own data
create an array of structures, write a matrix into each structure and reset it at a time usingFileWriteArray()
No not large, he explained on small numbers by 10: 8:2 vs. 6:4. But we have a lot of data.
How many samples can be considered representative for BP? I usually do not use less than 10000, small class it should have at least 1000
For trees, there is a wonderful graph of error depending on the number of trees.
So for sentinels, the number of trees more than 100 does not reduce the error.
The graph looks like this:
You can take different file sizes and get the answer to your question.
But this is not all.
For rf there is one more parameter "Sample Size". With it you can level out the difference in class. Anyway this parameter greatly affects the size of the error.
For example.
For Sample Size = 1000, 500 with file size 7500 lines balanced by classes looks depressing. But it is at first glance and not necessarily depressing. Suppose class "0" is long and class "1" is "out of market" . The error for "out of market" is over 0.5, which means that we are buying from scratch. If we switch places, it means that the "out of market" strategy tries to guess, but the "long" error is very small.
And the same file with Sample Size = 1000, 1000 looks very decent.
create an array of structures and write a matrix into each structure, if necessary, and reset it at one time using FileWriteArray()
I don't know beforehand how many columns there are... and aren't arrays of structures with dynamic arrays inside written to files? ) This is kind of a mess...
I just need to save a 2-d array, which has an unknown number of columns
exactly the same way ensembles of neural networks are built on bagging and you get the same thing as a forest
You have to understand that a forest is just a special case of bagging, instead of trees you can put anything, any weak models. A single tree is an elementary model that can't suppress anything
the forest "sort of" doesn't retrain because of random sampling, but in fact it retrains very easily and unpretentiously
both need to be regularized either through Decay (gradient step), early stopping, or through the r parameter of the forest, or do preprocessing, but all these improvements are usually within 5-10%. On bad data both models will work equally bad
about boosting (extreme, not GBM) they say it doesn't retrain too much, you should look it up, i can't say anything
https://www.analyticsvidhya.com/blog/2016/03/complete-guide-parameter-tuning-xgboost-with-codes-python/
Everything is about the same: rf, xgboost, SVM, GLM, nnet.
In some areas one model is better than the other, in others worse - all units of percent.
I get the impression that the model error is really the error of the predictor-target variable pair. There is a certain limit, above which you can't go by any tricks, but you can easily ruin a promising pair.