Machine learning in trading: theory, models, practice and algo-trading - page 2550
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Generally speaking, after training (on train) there is not one model, but a set of them, determined by meta-parameters. For example, different degree of the interpolation polynomial or different regularization coefficients in lasso regression, etc. Then the best value for the metaparameter is determined (the best model from the set is taken by testing). In turn, optimization of meta-parameter on test can also be determined by some parameters (meta-parameters), for optimization of which you can apply exam. For example, in what proportions to divide the original sample into train and test.
But, most likely, I just do not understand your idea.)
The best way is not to select meta-parameters from a single test plot, but from several glued together by cross validation or rollicking forward. Recently discussed.
I agree. Actually, I just wanted to express the thought that cross validation can also be arranged unevenly and be set with some of its own parameters (meta-parameters) which can also be optimized by another sample (instead of taking them from the ceiling).
I agree. Actually I just wanted to express the idea that crossvalidation can also be arranged in a complicated way and be set by some own parameters (meta-parameters), which can also be optimized by another sample (instead of taking them from the ceiling).
Correctly understood - to carry out any actions so that the sample would be more like a sample of exam.
The question is how best to do it then.
One option, which is often used, enumeration of predictors - but with a large set of them too much. And can we, say, compare strings (set of predictors) for similarity and robustness of the target outcome in the two samples? Then we can drop anomalous (say, rare or not found in sample exam) strings from training sample, and due to this we can train much better, in theory.
Correctly understood - to carry out any actions in order to make the sampling more like a sampling of exams.
The question is how best to do it then.
One option that is often used is to enumerate predictors - but with a large set there are too many of them. And can we, say, compare strings (set of predictors) for similarity and robustness of the target outcome in the two samples? Then we can drop anomalous (say, rare or not found in sample exam) strings from training sample, and due to this learn much better, in theory.
Vladimir had an article about it, I don't remember the name of the package, but it was just dropping unpredictable strings from the sample
Interesting, I'll have to look it up. But I want to throw out not the unpredictable ones, but the ones that don't occur in the sample outside of training. It is even interesting to mark them in some way, of course, and not just throw them out - to identify.
Interesting, I'll have to look it up. But, I want to throw out not what is not predictable, and those that do not occur in the sample outside of training. It is even interesting to mark them in some way, of course, and not just throw them out - to identify.
Perhaps we are talking about removing outliers in observations, which may be of two types - by response size and by predictor size (they may both be combined in the same observation). The outliers proper are called the former, while the latter are usually called something else. This science is well developed for linear regression. Probably, you can check each test observation to see if it is an outlier in some sense relative to an exam.
Interesting, I'll have to look it up. But, I want to throw out not what is not predictable, and those that do not occur in the sample outside of training. It is even interesting to mark them in some way, of course, rather than just throwing them out - to identify them.
Vladimir had an article about it, I don't remember the name of the package, but it just kicked out unpredictable strings from the sample
TheNoiseFiltersR package.Take a look at the article.
The best way is not to pick meta-parameters from a single test plot, but from several glued together by cross validation or rollicking forward. We discussed this recently.
And how to use CV results correctly? to take the best found model parameters and teach them to the whole dataset, or to use it only for good datasets selection
For example, the output is the following table
NoiseFiltersR package.Take a look at the article.
If everything is more or less clear with the noise predictors, not so much with the noise examples. I would like to know more about the methods of their definition (in the sense of theory, not the names of packages/functions used, although of course R always has links to articles). It is clear that there should be a "don't trade" class when classifying, since striving to be in the market all the time is considered a mistake. But it is not quite clear how this class may be described correctly in a more or less formal way.