Machine learning in trading: theory, models, practice and algo-trading - page 1848

 
mytarmailS:

This is not an idea to think about, but an idea to read!

text classification - to understand what and how to do and what algorithms

Sanity Analysis: for recognizing "positive/negative" speech.

This has been implemented in ready-made packages for 5-10 years, so you can start using them and don't have to keep reinventing the wheel.

Yes, I agree that the idea is for reading.
Yes, there are algorithms for speech classification.
The idea was to think about how to put it all together.

If you know any packages, please tell me their names.

 
Roman:

Yes, I agree that the idea is for reading.
Yes, there are algorithms for speech classification.
I meant to think about how to put it all together.

If you know any packages, please tell me their names.

There are hundreds of them, what can I tell you? Formulate your problem first.


text mining

santiment analysis

 
mytarmailS:

There are hundreds of them, what can I tell you?

text mining

santiment analysis

Well, at least here are the names of what to start with, and what to look for. Thank you.

 
Mihail Marchukajtes:

But the indicator, which is built the same seems rnormal, but when passed into the Expert Advisor is shifted by one bar and the result is a complete mess. If I can change it I would be grateful.

And how is the AI not collected at the opening of a bar for the last bar? If so, the shift is logical.

 
elibrarius:
Thank you. I used to use Precision, calling (for myself) it Accuracy for the class. I'll call it by common terms now).
And in general, Precision can be considered a basic metric when you have a "wait" class. Errors in Precision are direct losses from misclassification.
And Recall means lost profits, i.e., we waited instead of acting.
The bottom line is to maximize F1, which will find the best value with the minimum of prediction errors and the minimum of missed profit.

If we talk about bootstrapping(CatBoost), usually Precision goes up very fast, but Recall does it slowly, and while Recall goes up, Precision goes down. I mean that when you are training it would be good to control two indicators separately, let's say, setting limits 80<Precision>60 and Recall>50 and try to stop training in these boundaries. Different coefficients, such as F1, it is more difficult to do. It's a pity the developers didn't foresee this possibility, and I haven't yet figured out how to trim trees of the finished model.

Another idea is to divide the sample into sections of 10 and train 10 models in each section, see the behavior of the model in each section and trim the model trees, the average overtraining rates, and then somehow bring all the models together. In this way overtraining of models will be cleaned up and only data with stable trend information will be used.

 
Aleksey Vyazmikin:

And how is the OI not collected at the opening of the bar for the last bar? If so, the shift is logical.

No it is written every tick
 
Mihail Marchukajtes:
No it is written every tick

It is unclear how you request the data from the indicator in the Expert Advisor, if it is being built correctly on the chart. Have you checked the construction of the indicator in the visual mode of the strategy tester?

 
Aleksey Vyazmikin:

There is also an idea to divide the sample into sections of 10 and train 10 models in each section, see model behavior in each section and prune model trees, average overtraining rates, and then somehow bring all models together. In this way overtraining of models will be cleaned up and only data with stable trend information will be used.

I had a similar idea too, but for now I'm busy with something else. I hope to experiment with it soon.
It also has the disadvantage that the model will learn on a 10 times smaller data set. I think its generalizability will thereby decrease.

 
Aleksey Vyazmikin:

It is unclear how you request data from the indicator in the Expert Advisor, if it is built correctly on the chart. Have you checked the construction of the indicator in the visual mode of the strategy tester?

It is standard through Icon... I don't think it reads data from the file with every tick, but indicator reads it when new bar appears and it turns out that it takes slightly different value.
 
Roman:

Well, at least here are the names of what to start from, and what to look for. Thank you.

It's not names, it's links to everything you need, ready-made... and more.

Reason: