Machine learning in trading: theory, models, practice and algo-trading - page 2262

 
mytarmailS:

What do you mean?

just shake the matrix cow. it will be random ....

you need to know the goal - for what to shake, what the final res. should be ???

the goal is profit :D

Here you see all some scalpers in the market offering to sell ticks, and I have purely sporting interest
 
Maxim Dmitrievsky:

purpose - profit :D

Regarding the generation approach itself, a criticism from me)

When you create data and go through models looking for a model that will work on "new data" , do you understand that it's a fitting ? Do you realize that it's a fitting?

Since this "new data" is involved in the choice of the model, it is not"new data"... It's not very obvious but it's true !

We need to add a third sample, which was not involved in any way, have you done such a thing?


About the covariance, I can "shake it" but I'm not an expert in GMM.

I have a fake matrix

XX
     [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,]  0.7 -0.2  0.0 -1.7 -2.6  1.3 -0.4  0.9  0.4  -1.6
[2,] -0.7 -0.8 -1.4  1.5 -1.1 -0.1 -0.6 -0.4  1.0   0.2
[3,] -0.3  0.5  2.1  2.4  0.8 -0.3  1.3  1.3  0.2   0.4
[4,]  0.0  0.1 -0.1 -1.8 -0.4 -0.6  0.9  0.7 -1.2   0.9
[5,]  1.0 -0.6 -0.5  0.0 -0.3  1.2  2.3 -1.9  0.3   1.4

I created a GMM model

Here's the output of the model.

Model$parameters
$pro
[1] 0.2 0.2 0.4 0.2

$mean
               [,1] [,2]  [,3]           [,4]
 [1,]   7.00000e-01 -0.7 -0.15   1.000000e+00
 [2,]  -2.00000e-01 -0.8  0.30  -6.000000e-01
 [3,] -7.41241e-145 -1.4  1.00  -5.000000e-01
 [4,]  -1.70000e+00  1.5  0.30 -8.061356e-177
 [5,]  -2.60000e+00 -1.1  0.20  -3.000000e-01
 [6,]   1.30000e+00 -0.1 -0.45   1.200000e+00
 [7,]  -4.00000e-01 -0.6  1.10   2.300000e+00
 [8,]   9.00000e-01 -0.4  1.00  -1.900000e+00
 [9,]   4.00000e-01  1.0 -0.50   3.000000e-01
[10,]  -1.60000e+00  0.2  0.65   1.400000e+00

$variance
$variance$modelName
[1] "EEI"

$variance$d
[1] 10

$variance$G
[1] 4

$variance$sigma
, , 1

       [,1]  [,2]  [,3]  [,4]  [,5]  [,6]  [,7]  [,8]  [,9] [,10]
 [1,] 0.009 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
 [2,] 0.000 0.016 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
 [3,] 0.000 0.000 0.484 0.000 0.000 0.000 0.000 0.000 0.000 0.000
 [4,] 0.000 0.000 0.000 1.764 0.000 0.000 0.000 0.000 0.000 0.000
 [5,] 0.000 0.000 0.000 0.000 0.144 0.000 0.000 0.000 0.000 0.000
 [6,] 0.000 0.000 0.000 0.000 0.000 0.009 0.000 0.000 0.000 0.000
 [7,] 0.000 0.000 0.000 0.000 0.000 0.000 0.016 0.000 0.000 0.000
 [8,] 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.036 0.000 0.000
 [9,] 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.196 0.000
[10,] 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.025

, , 2

       [,1]  [,2]  [,3]  [,4]  [,5]  [,6]  [,7]  [,8]  [,9] [,10]
 [1,] 0.009 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
 [2,] 0.000 0.016 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
 [3,] 0.000 0.000 0.484 0.000 0.000 0.000 0.000 0.000 0.000 0.000
 [4,] 0.000 0.000 0.000 1.764 0.000 0.000 0.000 0.000 0.000 0.000
 [5,] 0.000 0.000 0.000 0.000 0.144 0.000 0.000 0.000 0.000 0.000
 [6,] 0.000 0.000 0.000 0.000 0.000 0.009 0.000 0.000 0.000 0.000
 [7,] 0.000 0.000 0.000 0.000 0.000 0.000 0.016 0.000 0.000 0.000
 [8,] 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.036 0.000 0.000
 [9,] 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.196 0.000
[10,] 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.025

, , 3

       [,1]  [,2]  [,3]  [,4]  [,5]  [,6]  [,7]  [,8]  [,9] [,10]
 [1,] 0.009 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
 [2,] 0.000 0.016 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
 [3,] 0.000 0.000 0.484 0.000 0.000 0.000 0.000 0.000 0.000 0.000
 [4,] 0.000 0.000 0.000 1.764 0.000 0.000 0.000 0.000 0.000 0.000
 [5,] 0.000 0.000 0.000 0.000 0.144 0.000 0.000 0.000 0.000 0.000
 [6,] 0.000 0.000 0.000 0.000 0.000 0.009 0.000 0.000 0.000 0.000
 [7,] 0.000 0.000 0.000 0.000 0.000 0.000 0.016 0.000 0.000 0.000
 [8,] 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.036 0.000 0.000
 [9,] 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.196 0.000
[10,] 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.025

, , 4

       [,1]  [,2]  [,3]  [,4]  [,5]  [,6]  [,7]  [,8]  [,9] [,10]
 [1,] 0.009 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
 [2,] 0.000 0.016 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
 [3,] 0.000 0.000 0.484 0.000 0.000 0.000 0.000 0.000 0.000 0.000
 [4,] 0.000 0.000 0.000 1.764 0.000 0.000 0.000 0.000 0.000 0.000
 [5,] 0.000 0.000 0.000 0.000 0.144 0.000 0.000 0.000 0.000 0.000
 [6,] 0.000 0.000 0.000 0.000 0.000 0.009 0.000 0.000 0.000 0.000
 [7,] 0.000 0.000 0.000 0.000 0.000 0.000 0.016 0.000 0.000 0.000
 [8,] 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.036 0.000 0.000
 [9,] 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.196 0.000
[10,] 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.025


$variance$Sigma
       [,1]  [,2]  [,3]  [,4]  [,5]  [,6]  [,7]  [,8]  [,9] [,10]
 [1,] 0.009 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
 [2,] 0.000 0.016 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
 [3,] 0.000 0.000 0.484 0.000 0.000 0.000 0.000 0.000 0.000 0.000
 [4,] 0.000 0.000 0.000 1.764 0.000 0.000 0.000 0.000 0.000 0.000
 [5,] 0.000 0.000 0.000 0.000 0.144 0.000 0.000 0.000 0.000 0.000
 [6,] 0.000 0.000 0.000 0.000 0.000 0.009 0.000 0.000 0.000 0.000
 [7,] 0.000 0.000 0.000 0.000 0.000 0.000 0.016 0.000 0.000 0.000
 [8,] 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.036 0.000 0.000
 [9,] 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.196 0.000
[10,] 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.025

$variance$scale
[1] 0.05824961

$variance$shape
 [1]  0.1545075  0.2746800  8.3090689 30.2834661  2.4721197  0.1545075  0.2746800
 [8]  0.6180299  3.3648296  0.4291874


$Vinv
NULL

What is a covariance matrix?

 
mytarmailS:

Regarding the generation approach itself, a criticism from me)

When you create data and go through models looking for a model that will work on "new data" , do you understand that it's a fitting ? do you realize that it's a fitting?

Since this "new data" is involved in the choice of the model, it is not"new data"... It's not very obvious but it's true !

We need to add a third sample, which was not involved in any way, have you done such a thing?


About the covariance, I can "shake it" but I'm not an expert in GMM.

I have a fake matrix

I created a GMM model

here is the output of the model

this is the covariance matrix ?

I checked it on the 3rd, yeah.

better to shake the centroids of the clusters. I.e. averages (means), each value. In a small range, where each value is the center of the normal distribution. After each shake, add samples.

That's a bad approach, don't bother. It's better to shake the average price increments, it's clearer, but longer

 
Maxim Dmitrievsky:

It is better to shake the average price increments, it is clearer, but longer

What is the fundamental difference?

 
mytarmailS:

What is the fundamental difference?

Nothing, it was an assumption that you can still shake it with minimal effort. But it will not work.

sorry, read it wrong. The difference is that gmm will find other clusters on modified increments. And by shaking the centroids, you're essentially doing nothing.

i.e. the goal is to generate rows with a different mean and or variance, similar to the original

but you take a small chunk of the original series (a couple of months) and generate the rest.

 
Maxim Dmitrievsky:

Nothing, it was an assumption that you can still shake it with minimal effort. But it won't work that way.

sorry, read it wrong. The difference is that gmm will find other clusters on modified increments. And by shaking the centroids, you're doing whatever it takes, basically.

i.e. the goal is to generate rows with a different mean and or variance, similar to the original

but you take a small piece of the original series and generate the rest.

Isn't it easier to create a generator of pseudo prices which can generate anything, and choose its parameters so that it generates such a series

that will pass trace, test and validity checks.

 
mytarmailS:

Isn't it easier to create a pseudo-price generator that can generate anything, and choose its parameters so that it generates a series

that will pass trainee, test and validation tests.

We don't need anything, we need to inherit the peculiarities of the series, on which we're going to trade.

 
Maxim Dmitrievsky:

we don't need anything, we need to inherit the peculiarities of the series we are going to trade on

Look, if it passed all our criteria, it means it inherited everything, everything we thought it would and even everything we never thought it would and never would have included in the model...

 
mytarmailS:

Look, if it passed all our criteria, it means that it inherited all that is necessary, and what we could assume and even what we did not know and would never include in the model...

come up with such a generator )

 
Maxim Dmitrievsky:

think of such an oscillator )

I'm not a genius))

Everything has already been invented... Same GMM, we take averages and change them as we wish up to the result, or change the series itself, or synthesize a spectrum and use it to reconstruct the signal, or... or...

Reason: