Machine learning in trading: theory, models, practice and algo-trading - page 1289

 
Maxim Dmitrievsky:

You can just run around in a circle, waving your arms, the result will be about the same.)

Actually, I'm talking about cycles. And mental experiments cost nothing, but can be very useful).

By the way, if there were real cycles in the market, they could always be distinguished by spectral analysis. So, at completely different analysis durations - pure noise spectrum. I did it long ago, if I'm going to repeat it with Python, I will demonstrate.

 
Maxim Dmitrievsky:

Well, what are the final errors? for both models and how it works on the new data

Man, there's a lot of deep analysis to be done here

The error delta is output, i.e. how much this predictor contributes to the overall model error. On the training domain removing one changed the error by no more than 0.5% (since this domain has a strong fit), while on the validating domain the error changes to 5.5% (since this domain was not adjusted). Permutation on both the traine and the valide jumps up to 5-6%.


In general, random re-training + random knot shuffling also gives random results.

I think I'll build up predictors by 1. I think up to 10-20 will quickly generate models, and then will slow down.

 
elibrarius:

The error delta is displayed, i.e. what contribution this predictor makes to the total model error. At the training section removing one changed the error by no more than 0.5% (because the fit was strong at this section), while at the validating section the error changes up to 5.5% (because this section was not adjusted). Permutation on both the traine and the valide jumps up to 5-6%.


In general, random re-training + random knot shuffling also gives random results.

I think I'll build up predictors by 1. I think up to 10-20 will quickly generate models, and then will slow down.

If the parameters vary slightly, it's normal, the forest is random. If you make a setside, it should always be the same.

I.e. both there and there ~5% on validation, it means they do the same job.

 
elibrarius:

is there a setseed in the algleeb?

there's a gcj for splits

I just put MathSrand() in the Expert Advisor or script or whatever before training the forest or during initialization

I think it even worked... I forgot... I don't do it now because I don't need it
 
Yuriy Asaulenko:

Actually, I'm talking about cyclicality. And mental experiments don't cost anything, but they can be very useful).

By the way, if there were real cycles in the market, they could always be distinguished by spectral analysis. So, at completely different analysis durations - pure noise spectrum. I did it long ago, if I'm going to repeat it using Python, I will demonstrate.

Why? Everything is clear anyway.

There are cycles in the market, but they are non-periodic, i.e. they appear, continue and disappear. Probably, they appear at random. The question is how to catch this phenomenon (and if it is possible).

 
Maxim Dmitrievsky:

Why? Everything is clear enough.

There are cycles in the market, but they are non-periodic, i.e. they appear, continue and disappear. Probably in a random way. The question is how this phenomenon can be caught (and if it is possible).

There are cycles in the market, but they are non-periodic - Maxim, cycles, by definition, are periodic, or close to it - pseudo-periodic. There are no other cycles.) Or they are not cycles).

It is impossible to predict or catch random phenomena. One can try to detect them only in the process of their occurrence and signs of life, in the course of the play, with no more than some probability, possibly very small.

 
Yuriy Asaulenko:

There are cycles in the market, but they are non-periodic - Maxim, cycles, by definition, are periodic, or close to it - pseudo-periodic. There are no other cycles.) Or they are not cycles).

It is impossible to predict or catch random phenomena. You can try to detect them only in the process of their occurrence and signs of life, in the course of the play, with no more than some probability, perhaps a very small one.

How to explain... in short, there is a signal or something that takes the form of a cycle, and after a bifurcation point it takes the form of another cycle. Or bifurcation point, whatever you want to call it, the meaning does not change

I don't know what it is called scientifically, but after a certain half-period the cycle can be "run" with a decent probability and low error, until it breaks again and a certain time (half-period or whatever it is called) passes, for which you can catch on again. This is kind of how it works in theory.

 
Maxim Dmitrievsky:

how to explain... in short there is a signal or something that takes the form of a cycle, and after a bifurcation point it takes the form of another cycle. Or bifurcation point, whatever you want to call it, the meaning does not change.

I don't know what it is called scientifically, but after a certain half-period the cycle can be "led" with a decent probability and low error, until it breaks again and a certain time (half-period or whatever it is called) passes, for which you can catch on again

If you know this, and it really does exist (let's call it: the development cycle of a phenomenon, which again is a regularly recurring event), then you can easily use it.

I can only see this kind of thing on history, when things have already happened. On real-time, I'll pass.) By the way, it's common for us to identify a signal only after it's over. In signal processing this is often done.

 
Maxim Dmitrievsky:

how to explain... in short there is a signal or something that takes the form of a cycle, and after a bifurcation point it takes the form of another cycle. Or bifurcation point, whatever you want to call it, the meaning does not change.

I don't know what it is called scientifically, but after a certain half-period the cycle can be "run" with a decent probability and low error, until it breaks again and a certain time (half-period or whatever it is called) passes, for which you can catch on again. That's kind of how it might work in theory.

It fits easily into my theory about the market. Just someone with a lot of money turned on his algorithm for recruiting positions/conducting transactions, some big bank, maybe the Central Bank, of course this is not done quickly, but since this participant was dominant, and the market situation contributed to this, it was possible to find signs of his algorithm. Of course, after the participant stopped influencing the market, the signs stopped working. There are many such participants (maybe 100), their algorithms may overlap, but there is an assumption that they are similar (remember technical analysis and the requirements for the banks to justify their trading operations with this analysis (at least in Russia so)), and for this reason it makes sense to analyze a large sample, where one and the same algorithm was running many times, then there is a chance to understand how it works, to describe it by indirect signs, but the model must learn to identify it and not work at this time on noise, waiting for the algorithm, under which it is running. I think this would of course work even better on stocks and derivatives, I don't do MO on forex.

But in the end, we need to find 10 models that will describe the algorithm for the big money, and learn how to determine which algorithm is preferable at a given moment in time. Since the cycle of the algorithm can be a couple of days and it will probably be repeated for a not very long time, it's okay if we enter with a small delay, the main thing is to choose the right model for this algorithm.

We are all just small fishes, which can join the whale, the big market participant, on mutually beneficial terms.
 
Yuriy Asaulenko:

If you know this, and it really does exist (let's call it: the developmental cycle of a phenomenon, which again is a regularly recurring event), then you can easily use it.

I can only see this kind of thing on history, when things have already happened. On the real - pass). By the way, it is a common phenomenon when we can identify a signal only after it has finished. In signal processing this is often the case.

It exists on the history, I do not know how to algorithmize it.

Reason: