Machine learning in trading: theory, models, practice and algo-trading - page 1839

New comment
 
mytarmailS:

Ahahahaha )))) LIVE !!!

That's just the words of the Trickster. Hare plogiatiat!!!! Waiting for him with no patience....
 

I want to ask everyone's advice.
In the Darch package I found the following option of model evaluation:
We count the error on the track and on the oob sections.
Then the final error is counted as
err = oob_error * comb_err_tr + train_err * (1 - comb_err_tr);

where comb_err_tr = 0.62. The multiplier that controls the contribution of errors from the train and oob sections. If you set it to 0, the estimation is by train only. If you set it to 1, the estimation is only by oob.

0.62 means that the error from oob has a little bit more weight in total model estimation.

I used this formula for a while, but now I have doubts.

The error on oob usually has a limit, and on train if overtraining went on, it could go as low as 0.
Approximately like this: (green is the estimated error from the formula)




According to this formula, the error will continue to decrease due to the decreasing trn error. And will only stop dropping when trn stops decreasing. At the same time, when retraining has started, the error on oob will start to grow.
In my opinion, stopping learning when err by formula begins to grow is too late.
Even at the point where the error oob is minimal is not optimal either. Due to a lucky randomization we accidentally found a minimum on oob, but it could be a fit to oob.
Maybe we should take the minimum error on oob, and count it as a limit for the error on train? That is, stop teaching the model when the error on trn became equal to the best error on oob (where I drew the vertical line)? The error on the oob will be worse, but it won't be a fit for either train or oob.

 
elibrarius:

I want to ask everyone's advice.
In the Darch package I found the following option of model evaluation:
We count the error on the track and on the oob sections.
Then the final error is counted as
err = oob_error * comb_err_tr + train_err * (1 - comb_err_tr);

where comb_err_tr = 0.62. The multiplier that controls the contribution of errors from the train and oob sections. If you set it to 0, the estimation is by train only. If you set it to 1, the estimation is only by oob.

0.62 means that the error from oob has a little bit more weight in total model estimation.

I used this formula for a while, but now I have doubts.

The error on oob usually has a limit, and on train if overtraining went on, it could go as low as 0.
Approximately like this: (green is the estimated error from the formula)




According to this formula, the error will continue to decrease due to the decreasing trn error. And will only stop dropping when trn stops decreasing. At the same time, when retraining has started, the error on oob will start to grow.
In my opinion, stopping learning when err by formula begins to grow is too late.
Even at the point where the error oob is minimal is not optimal either. Due to a lucky randomization we accidentally found a minimum on oob, but it could be a fit to oob.
Maybe we should take the minimum error on oob, and count it as a limit for the error on train? That is, stop teaching the model when the error on trn became equal to the best error on oob (where I drew the vertical line)? The error on the oob will be worse, but it won't be a fit for either train or oob.

There is logic in this. The margin of error is determined by the probabilistic model, and both a reasonable sample size and the number of retraining have some optimal size, the increase of which does not improve the result

 
mytarmailS:

Trading system through the eyes of an algotrader

R - you're just apuenenen! :)

What about the digital filters or levels show anything interesting? :D

 
Maxim Dmitrievsky:

Do the digital filters or levels show anything interesting? :D

The last thing I did was to look for a pattern superposition...

We have a level - when the price crosses it, we fix this pattern and fix it as a training sample

Patterns can be different.

Moreover, at one moment there may be many patterns at the same time, that's the whole point, I'm looking for a clear pattern in this set of patterns that appeared in the moment, a clear set that solves something


For mining rules by patterns I use"associative rules" this approach differs from usual ones by the fact that each training example may contain any number of elements and doesn't take into account the orderliness of signs, which is also good as for me


the target - to find an extremum from which there will be an increase by 10 points

 x[i]==min(x[(i-1):(i+10)])

not the best solution, but that's what I'm writing about, so far only buy


The "apriori" mining algorithm from the "arules" package


this is how the found rules look like

inspect(head(rules.sorted,20)) 
     lhs                              rhs   support     confidence lift     count
[1]  {(28)(28)(-1);1,(44)(45)(-1)} => {BUY} 0.001017018 0.5769231  3.046559 15   
[2]  {(25)(23)(-1);1,(5)(3)(-1)}   => {BUY} 0.001084819 0.5517241  2.913491 16   
[3]  {(31)(33)(-1),(8)(6)(-1)}     => {BUY} 0.001084819 0.5000000  2.640351 16   
[4]  {(49)(45)(-1),(54)(52)(-1)}   => {BUY} 0.001017018 0.5000000  2.640351 15   
[5]  {(25)(23)(-1),(82)(84)(-1)}   => {BUY} 0.001017018 0.4838710  2.555178 15   
[6]  {(46)(48)(-1),(56)(56)(-1)}   => {BUY} 0.001017018 0.4838710  2.555178 15   
[7]  {(25)(23)(-1);1,(40)(41)(-1)} => {BUY} 0.001017018 0.4838710  2.555178 15   
[8]  {(29)(30)(-1),(37)(39)(-1)}   => {BUY} 0.001017018 0.4838710  2.555178 15   
[9]  {(34)(32)(-1),(76)(74)(-1)}   => {BUY} 0.001898434 0.4745763  2.506096 28   
[10] {(25)(22)(-1),(7)(6)(-1);3}   => {BUY} 0.001152621 0.4722222  2.493665 17   
[11] {(17)(16)(-1);1,(49)(45)(-1)} => {BUY} 0.001017018 0.4687500  2.475329 15   
[12] {(46)(48)(-1),(62)(60)(-1)}   => {BUY} 0.001017018 0.4687500  2.475329 15   
[13] {(20)(21)(-1),(45)(46)(-1)}   => {BUY} 0.001017018 0.4687500  2.475329 15   
[14] {(19)(18)(-1);1,(60)(57)(-1)} => {BUY} 0.001220422 0.4615385  2.437247 18   
[15] {(25)(23)(-1);1,(47)(45)(-1)} => {BUY} 0.001152621 0.4594595  2.426268 17   
[16] {(40)(41)(-1),(71)(71)(-1)}   => {BUY} 0.001152621 0.4594595  2.426268 17   
[17] {(2)(1)(-1);4,(6)(6)(-1)}     => {BUY} 0.001084819 0.4571429  2.414035 16


here's rule "1" (the best one) in action on the new data

without any tampering, as is, in the sequence as is...

decide for yourself, if this is an interesting topic or not

continuation of the same


Then it is possible to add AMO to these inputs as if "from above" to filter an entry/no entry.

There is also infinite potential to increase the number and quality of patterns

Maybe something cool will come out, but I don't have energy and fuse, I've gone on a creative drinking spree ((



I think the levels are the most promising tool for creating TS ...

In my opinion, the level is not a stupid fractal of Bill Williams, but an event at a specific price, most likely "a lot of moves".

 
mytarmailS:

Then you can add AMO to these inputs as if "from above" to filter to enter/not to enter.

There is also infinite potential to increase the quantity and quality of the patterns

Maybe something cool will come out, but I have no energy and no fuse, I'm on a creative drinking binge ((

I will read when I will have enough energy ())

 
Maxim Dmitrievsky:

I'll read it when I get the hang of it ))

google something shorter, there's essentially nothing to read.

 
Oh, how f-cked up... people don't drink, never drink, never, never....
 
mytarmailS:
Oh, how f-cked up... people don't booze, never booze, never, never....
I know what you mean. I'm coming off the second day myself :-)
 

I see that you are trying to find a pattern. It is as simple as three kopecks) It is the wave theory. But it's not in the public domain at the moment.

Imagine you've been married for 22 years. How likely is it that you will get divorced today or tomorrow? Teach machine learning this understanding and only then move on to the simpler questions - the financial markets.

I understand that I'm having a hard time approaching the topic.

Respectful of Yusuf's words. He was always correct about the continuity of history and the current moment. And the importance for the future.

1...183218331834183518361837183818391840184118421843184418451846...3494
New comment
Reason: