Machine learning in trading: theory, models, practice and algo-trading - page 352

 
Maxim Dmitrievsky:

How wrong, you are creating a classification model. The larger the sample, the stronger the generalization, the model becomes more stable in general and less accurate in particularities, respectively, the smaller the profit

Let's put it in terms of efficiency. If the system's efficiency drops as a result of complication, such a system is uninteresting. A decrease in the value of profit + loss indicates a decrease in efficiency.

With increasing stability the efficiency should increase. Efficiency can be defined as profit/(profit+loss) ratio. or KPI per trade.

 
Yuriy Asaulenko:

Let's talk in terms of efficiency. If the system's efficiency drops as a result of complication, such a system is uninteresting. A decrease in the value of profit + loss indicates a decrease in efficiency.

With increasing stability the efficiency should increase. Efficiency can be defined as profit/(profit+loss) ratio. There is also KPI per trade.


It is not applicable to CS)) it turns out that the long term efficiency increases and the short term one decreases) There is a strategy and tactics... When you train NS in a short term it wins tactically, and in the long term - strategically, different factors start to influence...
 
Yuriy Asaulenko:

Let's put it in terms of efficiency. If the system's efficiency drops as a result of complication, such a system is uninteresting. A decrease in the value of profit + loss indicates a decrease in efficiency.

With increasing stability the efficiency should increase. Efficiency can be defined as profit/(profit+loss) ratio. You can also take KPI per trade.

Let's skip the bicycles called KPI.

Econometrics uses information criteria.

Here's the definition

Informational criterion is a measure of relative quality of econometric (statistical) models used in econometrics (statistics) which takes into account the degree of "fitting" of the model to the data with an adjustment (penalty) to the used number of estimated parameters. That is, the criteria are based on a certain compromise between model accuracy and complexity.

Informational criteria are used exclusively to compare models with each other, without meaningful interpretation of their values. They do not allow testing models in the sense of testing statistical hypotheses. Usually the lower the values of the criteria, the higher the relative quality of the model.


If we take a package in which model selection is assumed, one of the information criteria will be used.

 
Dimitri:


Faa writes the right idea, but he's not stating it correctly.

You have a series and a set of predictors. You divide the series into three parts - training sample and forward (the simplest case).

You build, for example, 20 models.

The point is that a model from the list is selected not by the best on the training sample, nor by the best on the forward one. A model is selected that gives approximately the same quality score on both training and forward.

We're both right.

An informational criterion is used to select the simpler one, but the informational criterion by no means precludes forward testing.

 
SanSanych Fomenko:

Let's skip the bicycles called KPIs.

Econometrics uses informational criteria.

One absolutely does not interfere with the other and does not replace it. In economics, there are efficiency criteria in the sense that they absolutely correspond to the KPI. There is no need to invent anything.

In this case, we are evaluating the result, not the process. We no longer care about process criteria. The system is a black box. So we compare the characteristics of the boxes.

There are also evaluation criteria - effectiveness per 1 ruble of investment, the average efficiency of a deal, etc. The whole business works on such criteria. What are we doing in the end?

 
Maxim Dmitrievsky:
Okay. You have increased the stability of the system - you have removed the drawdown. Even if the total profit has not even dropped, it remains constant. The question is: What happened to the deals that allowed you to get out of those pits? If you had not removed them, only they would have increased the profit of the system.
 
Yuriy Asaulenko:
Okay. You have increased the stability of the system - you have removed the drawdown. Even if the total profit does not even fall, it remains constant. Question: What happened to the deals that allowed you to get out of the holes in the previous version? If you had not removed them, only they would have increased the profit of the system.

If you train the neuron on different time intervals, then you will have different results, there is nothing to compare... the signals will be different, the model will be trained differently and there is nothing to influence that. I didn't take anything away)
 
Maxim Dmitrievsky:

If you train the neuron at different time intervals then you will have different results, there is nothing to compare even... there will be different signals, the model will be trained differently and you can not affect it at all. I didn't remove anything)

It is clear that the optimizer has removed. But repeatability of results is a requirement of any experiment. I.e. if we have different results in the test when training on different sections, it's a reason to think about it.

Imagine training identical NS in speech recognition using the same ToR and then claiming that they can't even be compared, because they were trained differently.

 
Yuriy Asaulenko:

It is clear that the optimizer has removed. But repeatability of results is a requirement of any experiment. I.e. if we have different results in the test when training on different sections, it's a reason to think about it.

Imagine that we train identical NS in speech recognition using the same ToR and then claim that they can't even be compared, because they were trained differently.


Well sort of yes, but in the current model it is impossible, it is not so complex that it would approximate in the same way a small set of data and a huge
 
Maxim Dmitrievsky:

Well, sort of, yes, but in the current model it's impossible, it's not so complex that it would approximate in the same way a small set of data and a huge

In general, if we talk about minutes, the market is statistically homogeneous, i.e. statistics changes little (stable) from week to week, from month to month. I don't know, I haven't studied the question. As far as I remember you are working on 1 minute.

In principle, simple systems should converge faster if they are convergent. That is, they have one high maximum. By the way, we generate these highs ourselves using the "wrong" predictors.

Reason: