Is there a pattern to the chaos? Let's try to find it! Machine learning on the example of a specific sample. - page 12

New comment
 
elibrarius #:

Do you select these models based on the best one on the test?

Or among the many best on the test is the best on the exam as well?

Specifically there was selection simply by the best on the exam.

 
Aleksey Vyazmikin #:

Specifically there was selection simply by the best on the exam.

I showed the best on the exam too. There will be no exam before going into real trading. Or rather it will be for real money....

Now I made a selection of signs by valking forward (10000 through 5000 and one trayne like yours and one test), on the exam both of them merge.

It is necessary to do the selection somehow on the test, so that the learnability is preserved on the exam.

 
elibrarius #:

I showed the best one in the exam too. There will be no exam before going into real trading. Or rather, it will be for real money....

Now I made a selection of signs by valking forward (10000 through 5000 and one trayne like yours and one test), on the exam both merge.

It is necessary to make the selection somehow on the test, so that the learnability is preserved on the exam.

At the moment you can only increase the probability of correct selection, unfortunately. That's why I am considering batch trading, when many models are selected at once, hoping that the average accuracy will be sufficient and I will be able to get an average profit.

 
It is necessary to find working features out of hundreds of thousands of your features, and then understand why they work. And then you need to write different TCs on them not by bruteforcing, but by selecting optimal hyperparameters.
Otherwise, it will still turn out to be fitting, when you have to choose from hundreds of models according to the exam.
The most important thing is to understand why the features work, at least approximately. Then they can be improved, or labels to them.

Stacking a bunch of unclear models is not a good idea either. Because you will have to retrain a bunch of unknown things again.

You need bruteforce with feature selection to choose good ones and then meditate on why they work. Then it will become clear where to go next. Bruteforce itself is ineffective for TC preparation, it should be considered as an exploratory one.
 
Maxim Dmitrievsky #:
It is necessary to find working features out of hundreds of thousands of your features, and then understand why they work. And then you need to write different TCs on them not by bruteforcing, but by selecting optimal hyperparameters.
Otherwise, it will still turn out to be fitting, when you have to choose from hundreds of models according to the exam.
The most important thing is to understand why the features work, at least approximately. Then they can be improved, or labels to them.

Stacking a bunch of unclear models is not a good idea either. Because you will have to retrain a bunch of unknown things again.

You need bruteforce with feature selection to choose good ones and then meditate on why they work. Then it will become clear where to go next. Bruteforce itself is ineffective for TC preparation, it should be considered as an exploratory one.
I agree. Understanding of the process can be achieved from different angles)
 
Maxim Dmitrievsky #:
It is necessary to find working features out of hundreds of thousands of your features, and then understand why they work. And then you need to write different TCs on them not by bruteforcing, but by selecting optimal hyperparameters.
Otherwise, it will still turn out to be fitting, when you choose from hundreds of models according to the exam.
The most important thing is to understand why the features work, at least approximately. Then they can be improved, or the labels to them can be improved.

It's also not a good idea to pack a bunch of obscure models. Because then you will have to retrain a bunch of unknown things again.

You need bruteforce with feature selection to choose the good ones and then meditate on why they work. Then it will become clear where to go next. Bruteforce itself is ineffective for TC preparation, it should be considered as an exploratory one.

The point is that the task of understanding the reason of predictor efficiency is extremely difficult and lies in the field of market behaviour interpretation, or do you have a more reliable approach? Besides, predictors work in a group because they are primitives, and how to pull together predictors that work in a group is not a simple question, if it is bousting - so far the obvious is to use a decision tree. And to build effective decision trees, you need to significantly reduce the sample, and better already feed only those predictors that presumably form an effective relationship. And here the method of model search can be very useful, as the model uses, as a rule, only a part of predictors.

Fitting or not fitting - I think all probability-fitting actions are fitting. Another thing is that the history of the distribution of these probabilities over the predictors may be repeated, or it may be forgotten for a long period of time. And here it is important to have some method to determine the transition of these stages.

 
Aleksey Vyazmikin #:

Training what is called out of the box with CatBoost, with the settings below - with Seed brute force gives this probability distribution.

1. Sampling train

2. Test selection

3. Exam sample

As you can see, the model prefers to classify all almost everything by zero - so there is less chance to make a mistake.

Alexey, training is essentially fitting, isn't it?

 
Renat Akhtyamov #:

Alexei, training is essentially fitting, isn't it?

Essentially, yes.

Optimisation in a tester is about changing the metrics on which the algorithm operates, and learning in MO methods (trees and their variants, NS) is about changing the algorithm by evaluating and interpreting the history of the metrics.

Symbiosis, would be epic.....

 
Renat Akhtyamov #:

Alexei, training is essentially fitting, isn't it?

Teaching schoolchildren is also fitting their knowledge to existing knowledge)

 
Aleksey Vyazmikin #:

That's the point, the task of understanding the reason for predictor performance is extremely difficult, and lies in the realm of interpreting market behaviour, or do you have a more robust approach? Besides, predictors work in a group, because they are primitives, and how to pull together predictors that work in a group is not a simple question, if it is bousting - so far the obvious is the use of a decision tree. And to build effective decision trees, you need to significantly reduce the sample, and better already feed only those predictors that presumably form an effective relationship. And here the method of model search can be very useful, as the model uses, as a rule, only a part of predictors.

Fitting or not fitting - I think all probability-fitting actions are fitting. Another thing is that the history of the distribution of these probabilities over the predictors may be repeated, or it may be forgotten for a long period of time. And here it is important to have some method to determine the transition of these stages.

small groups of 5 to 10 to train.

1-3 is better.

If none of them produce anything, what's the point of talking about a mythical connection between them? rubbish + rubbish...

1...5678910111213141516171819...32
New comment
Reason: