Gathering a team to develop an IO (decision tree/forest) in relation to trend strategies - page 12

 
If a miracle happens and the team comes together, we will have to choose a learning algorithm and a method for evaluating the model.
 
Roffild:
If a miracle happens and the team comes together, we will have to choose a learning algorithm and a method for evaluating the model.

I can nibble on a reflection group.

 
Aleksey Vyazmikin:

...I'll have to think about an alternative site. Maybe someone knows a similar one? I'm thinking something like a board where you can share pictures and edit them somehow, a separate chat room, and something like a reservoir of clever ideas...

For example. They also have a repository for sharing code as well. They also have a mobile application. Imho, a very handy service.

Trello
Trello
  • trello.com
Выбросьте длинные цепочки электронных писем, устаревшие таблицы, не такие уж и клейкие стикеры и неуклюжие программы для управления проектами. Trello помогает увидеть все детали проекта с первого взгляда. Зарегистрироваться – Это бесплатно.
 
Roffild:

To assess the quality of the model (network or forest), error values of MSE, OOB, etc. are used.

But unlike in picture recognition, where a human is the verification sample, it is very difficult to make such a sample for a price chart. Therefore the estimation of a price model by MSE, OOB, etc. is often misinterpreted.

There is no clear definition of "overfitting" the model.

So I stopped checking the model by MSE, OOB etc.

I now prefer to overlay the training result on the price chart to see the whole picture.

Here is my method of evaluating the quality of the model (I have already posted this picture):


IMHO, what is needed is not a picture, but objective, quantitative metrics, and if we are talking about an understandable consumer evaluation of model quality for trading, then you can measure them by signal productivity, for example.


 
Dennis Kirichenko:

For example. A repository for code sharing is also connected. They also have a mobile app. Imho, very handy service.

Hm... thanks for the tip, very interesting service. Do you use the free version? You may connect as many extensions as you like?

 
Roffild:

Now I prefer to overlay the result of the training on the price chart to see the whole picture.

Here is my method of evaluating the quality of the model (I have already posted this picture):

Tell me how to read this chart in the basement. You have how many targets there (I just see a divergence of 4 points - 4 targets?), do I understand correctly that the predictions occur at the beginning of the bar (then why does the opening not match or am I reading the chart wrong?) by one bar?

Visualisation is useful for the thought process, but without expressing these divergences in numbers it is impossible to automate the model estimation process for the same fitness function.

 
Roffild:
If a miracle happens and the team comes together, you will have to choose a learning algorithm and a method for evaluating the model.

Are you in the team or not?

 
Алексей Тарабанов:

I can do a bit of a reflection group.

OK, let's put it that way...

 
Dennis Kirichenko:

For example. A repository for code sharing is also connected. They also have a mobile app. Imho, a very handy service.

Thank you, I will have to study the service. Or have a look at the existing projects based on it.

 

I will share my thoughts on the evaluation of MoD models.

I don't know if there is such a thing as a herbarium in MO, but I will continue to use it. In case anyone does not understand, a herbarium is a collection of good leaves from trees, and one leaf or many can be collected from one tree. This model has a disadvantage when voting - the number of leaves describing one phenomenon (target) at different moments of time will be different, i.e. if we represent sampling as a field, it turns out that the leaves are scattered in different sets on the field, which affects the quality of the vote. So I think that to estimate this model (the method works for forests but it's more primitive, the emphasis should be put only on predictive ability in the field) it is necessary to present each leaf (tree) as a layer, overlay these layers on each other and where the leaves overlap calculate average predictive ability by adding factor that affects product depending on number of leaves (not necessary for forests), then look on resulting map and estimate its uniformity. Such a map can be evaluated in different ways, by adding the third space - by vertices, or by using Kohonen map method - by colour - for clarity, and to find the overall mean and RMS of this map. Then we can see the quality of the model, how strong its predictive power is over the whole sample, not just the aggregate. Such an estimate could help the fitness function look for leaf/tree improving areas of the sample with low average predictive power.

What do you think?

Or am I not making myself clear again?

Reason: