Machine learning in trading: theory, models, practice and algo-trading - page 3671

 
Aleksey Nikolayev #:

Imho, it depends on the way of building trees (stopping splits). The stopping can be done just on the basis of leaf sizes.

In any case, a more or less meaningful theoretical answer can be obtained with the help of Monte Carlo. A slightly more practical answer is probably with the help of cross-validation.

Yes, I think we should try to make the leaves close in size. But the leaves in which random is collected will still be large, because they just contain examples with small price movements, and these are the majority. And it makes no sense to miniaturise the random - you will also get random.
.

I think that if the size of leaves differs not more than 2-3 times, then weighting is rather useful (theoretically). But there is no guarantee that you won't get huge leaves with rubbish.

 
And without exploratory analysis, how do you figure out what leaves to make and what will come out of it?)
 
Forester #:
Yes, I think that we should try to make the leaves close in size. But the leaves in which random is collected will still be large, because they just contain examples with small price movement, and these are the majority. And it makes no sense to miniaturise the random - you will also get random.
.

I think that if the size of leaves differs not more than 2-3 times, then weighting is rather useful (theoretically). But there is no guarantee that you won't get huge leaves with rubbish.

It's pointless to consider building a model without considering preprocessing and how to divide by checkout.

1. Preprocessing. Where does this amount of randomness come from? We have to work with the selection of predictors. The source of classification error is not the way the tree is constructed, but the fact that the same predictor value gives one class in some cases and gives a different class in other cases. It is meaningless to classify noise.

2. On what basis is the value of 0.5 being considered. Do you have a class probability distributed normally with a matching mean and median? How do you divide into classes? There are a large number of ways to divide into classes, the most primitive and dubious is by halves.

 
СанСаныч Фоменко #:
classes

As I understand it, Forester is talking about regression, not classification.

 
Aleksey Nikolayev #:

As I understand it, Forester is talking about regression, not classification.

In the end, there are always classes: "buy-sell" or more diverse but classes. And when predicting value, the same question stands: at what probability of predicting value to make a decision, e.g. to buy.

But when predicting value, the situation has a different complexity: the probability must be accompanied by a confidence interval, which I didn't see in the post, so I decided on classes.

 
СанСаныч Фоменко #:

It is pointless to consider building a model without considering preprocessing and how to divide into checkouts.

1. Preprocessing. Where does this amount of randomness come from? We have to work with the selection of predictors. The source of classification error is not the way the tree is constructed, but the fact that the same predictor value in some cases gives one class, and in other cases gives another class. It is pointless to classify noise.

2. On what basis is the value of 0.5 being considered. Do you have a class probability distributed normally with a matching mean and median? How do you divide into classes? There are a large number of ways to divide into classes, the most primitive and dubious is by halves.

I mentioned 0.5 as an example of a noisy sheet with no profit.
The probability (value of the sheet) in the 0.5 classification is equivalent to the equality of income and expenses in the regression, i.e. the mean of the sheet will be = 0. I.e. by talking about 0.5 in classification this is me making a translation from 0 in regression.

And the 0.5 classification or 0 regression comes from the Distribution of Real Market Gains


Most movements are small and poorly predictable. So they will pile up in several sheets anyway. And it will be a great success if not in all of them, then the model will not be able to predict anything at all. You have to subtract the markup from them, which will only worsen the situation.

But still there are successful leaves with reduced amount of noise, and the task is not to drown them out by averaging with noise leaves from other trees during averaging.
In general, there is already a solution to this - to try to make leaves closer in size, if weighting by the number of examples is applied.

Well, how to choose the threshold for the action (buy|sell) is another issue. It deserves a separate consideration.

 
Classification is a regression with sigmoid on the window. Some algorithms take values normalised to the range 0;1.
Noisy predictions are clustered closer to zero. You can remove them by privacy levelling if there is less than 50% noise in the data. If it is more than that, nothing will help. The idea will not work.
 

I don't really know what Anokhin does or why he does it, but here it is.


 
mytarmailS #:
Recipe for classification GUARANTEED without retraining

Take the class labels

Predictors you pass through unmap then into hdscan then look for such clusters in the sets of which the same class label is often found....

That's it... :)

The classifier will work fine on new data

I made it without labels at all. Cluster numbers are class labels, i.e. all clusters are searched and two are selected: one to buy and one to sell.

Stable, but noisy.

Comparison with my method:


 
Maxim Dmitrievsky #:

I made it without labels at all. Cluster numbers are class labels, i.e. all clusters are searched, two are selected: one to buy and one to sell.

Stable, but noisy.

Comparison with my method:


Deals are few. And you can't see their distribution over time.
Maybe it's like this:
Here is an illustrative example with 3386 deals for 5 years:
Number_vs_Time
The same deals are placed on both charts.
The upper chart is built with an even step between deals (as MetaQuotes tester shows now). It looks promising.
The lower chart is built on a time scale and shows that the Expert Advisor has traded only a few weeks in 5 years. 1 pixel horizontally = 1 week.
According to the lower chart, it is immediately clear that there will be 2 years of inactivity in the beginning, then the Expert Advisor will trade for a few weeks and then 2.5 years of inactivity again. Such trading is no longer as interesting as the impression from the first chart.
Is there no time scale chart builder in Python? It doesn't seem to be a complicated thing... I draw my own charts in my browser.