Machine learning in trading: theory, models, practice and algo-trading - page 2589
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
https://stats.stackexchange.com/questions/31513/new-revolutionary-way-of-data-mining
When choosing a model it is suggested to optimize not by profit on OOS, but by the ratio of this profit to profit on the tray. Or throw out models with small this ratio and from the remaining to take the maximum profit on the OOS. This is if you understand the quotes literally, without speculation.
When selecting a model it is suggested to optimize not by profit on OOS, but by the ratio of this profit to profit on the tray. Or throw out the models with small such ratio and take the maximum of the remaining ones in terms of profit on the OOS. This is if you understand the quotes literally, without speculation.
When selecting a model it is suggested to optimize not by profit on OOS, but by the ratio of this profit to profit on the tray. Or throw out the models with small such ratio and take the maximum of the remaining ones in terms of profit on the OOS. This is if you understand the quotes literally, without speculation.
In my above example with coins and 10000 people. Let heads be 1, tails 0. If we act on this algorithm, we won't get anything either. This is understandable for the described context. In other words, if we come across some edge, then it's not so important whether we take profit ratio on IS and OOS or something else, and if there is no edge, then none of these methods will work.
Exactly! We need to assess the presence of edge first. And then think about how to select. For example, we can do the following: we look at the IS share of models by some metric above a certain threshold. For example, the win rate is higher than 55% - 45% of models. Ranking by win rate, we take some TOP. Then we check results of OOS for this top. Out of selected models win rate higher than 55% is given by the same 45% (ratio of models giving such win rate at OOS to all selected)? - I think this group of models can be safely thrown out. If you can see that such a selection of the TOP works, it means there is an edge, and by how strongly this effect is expressed you can estimate the quality of the pattern. It is decided that it is strong enough. All, then it is a matter of technique to select - even if by the same win rate, PF, you don't have to bother with complicated metrics and logics, and by win rate and PF directly on the IS.
Alexey, can you give me a piece from the quote, where it says about profit, profit maximum, throwing out models....
I have a free translation) The point is that initially many models are trained and in the end you have to choose a working one (model evaluation). Comrade claims that everyone usually chooses the model that just gives the maximum result on the OOS and this is the wrong approach. His second quote states how it should be done.
You know you are doing well if the average for the out-of-sample models is a significant percentage of the in-sample score. This is translated as maximizing the ratio of profits on the OOS to profits on the track.
Generally speaking, you are really getting somewhere if the out-of-sample results are more than 50 percent of the in-sample. This can be translated as discarding models where the ratio of profit on OOS to profit on tray is less than 0.5Well, it's kind of a question of selecting models, yes, as in optimization. You can come up with your own subjective criteria.
It is not bad if there is a bunch of models with slightly different parameters, that is, allowing for variation, but they all pass the OOS. But it is not a panacea of course.
Alexey, are there any techniques to restore the optimization surface?
Supplement model quality metrics for missing incoming, conditional, sets of hyper parameter values? Well boosting is simple to teach. What would that be needed for?
Supplement the model quality metrics for the missing incoming, conditional, sets of hyper parameter values? Well, it's easy to teach the boosting. And what can you need it for?
Alexey, are there any techniques for surface optimization reconstruction?
In the model parameter space? That's a huge dimension. This is possible only for very simple models with a small number of predictors.
It is not very clear how it is possible to build a surface in a space of huge dimensionality. We simply have very few points in comparison with this dimensionality. Unless by some dimensional downscaling visualization like PCA etc, but the point is unclear.
Well, it's kind of a question of selecting models, yes, as in optimization. You can come up with your own subjective criteria.
It is not bad if there is a bunch of models with slightly different parameters, that is, allowing for variation, but they all pass the OOS. But this is not a panacea of course.
Earlier you had an idea of combining standard and custom metrics, which I understood as follows: the models are trained using standard ones, while the selection is done using custom ones.