Machine learning in trading: theory, models, practice and algo-trading - page 3007

New comment
 
How I determined for myself which input data is good and which is not: forward behaviour during retraining. If the balance sheet immediately rolls to the bottom of the deepest gorge - phi, the data is not good. Usually it's the difference between closing prices in chronological order. Slightly better (not immediately to the bottom, but wiggling), if you mess around and add the difference between high/low, etc.

The best ones in my memory (when the balance tried to go up) are when you fed the number of closed candles up and down, rather than some difference or indicator readings. Tried 6500 candles (one year) - weak reaction, no changes (both there and 3200 on average). I tried 10 - too much change, the price can close up/down 20 times. I tried 100 and it was the optimal variant.
Then, when retraining, the balance tried to go up on the forward. But, I was not convinced, I wanted more.

I agree with Alexei Nikolaev, about 15 years ago I studied the three-wave from the critics of the VTE, and it turned out to be the most logical - nothing divides the price into a matryoshka like impulse-correction-impulse or ABC.

Now I think to try to feed neural networks with the input prices of the beginning and the end of each of these movements and the difference between them with the number of bars (minute bars), in order to somehow "draw" the neural network a picture of these movements on the chart
 
Slava #:

The control example does not fit.

Categorical cross-entropy is used in classification models where there are more than two classes. And after softmax. Softmax converts a set of values into a set of probabilities whose sum is 1.

Try a control example like this:

pred: 0.1, 0.1, 0.2, 0.5, 0.1

true: 0, 0, 1, 0, 0, 0

The example I gave is just from the categorical cross-entropy section (and you apparently didn't notice that there the sum of values is 1 in each instance). The fact that it does not work as in Keras is an indicator for me, which means that either the implementation or the description of CCE in MQL5 does not match the expected. Then a detailed description is required. By the way, in pytorch CrossEntropyLoss includes a preliminary softmax inside. But in general, since the documentation on matrices in MQL5 contains the idea that the interface is similar to the python one, the coincidence of behaviour is implied. And if there is no coincidence, it causes problems and perplexity.

Having many classes implies working with matrices (when we have a bunch of samples/rows, each of which has classes), so your example with a vector still does not answer the question.

 
Aleksey Nikolayev #:

I haven't thought about it, but I think it's unlikely, as the order of movements is important in prices.

Just in case, a picture to illustrate Mandelbrot's idea. Each price movement, if possible, is split into three movements (by selecting the maximum correction within it), then it becomes a node of the tree. If there is no correction inside the movement (or it is less than a given value), then it becomes a leaf of the tree.


Isn't this cooler?

Ramer-Douglas-Pecker algorithm - Wikipedia (wikipedia.org)

The same algorithm will help you break down historical data into trend and flat segments.

a poet's dream, actually....

because

any econometric theory can be applied.

and everyone's favourite neura

 
Renat Akhtyamov #:

Isn't that cooler?

Ramer-Douglas-Pecker algorithm - Wikipedia (wikipedia.org)

the same algorithm will help to split historical data into trend and flat segments.

a poet's dream, proper....

because.

any econometric theory can be applied already

and everyone's favourite Neura.

That's cool.

I'll have to look into it.

 
reinvented resampling
 
Ivan Butko indicator readings. Tried 6500 candles (one year) - weak reaction, no changes (both there and 3200 on average). I tried 10 - too much change, the price can close up/down 20 times. I tried 100 and it was the optimal variant.
Then, when retraining, the balance tried to go up on the forward. But, I was not convinced, I wanted more.

I agree with Alexei Nikolaev, about 15 years ago I studied the three-wave from the critics of the VTE, and it turned out to be the most logical - nothing divides the price into a matryoshka like impulse-correction-impulse or ABC.

Now I think to try to feed neural networks with the input prices of the beginning and the end of each of these movements and the difference between them with the number of bars (minute bars), in order to somehow "draw" the neural network picture of these movements on the chart
Well, this is pulling an owl on a globe. First a certain strategy is thought up, then the NS is trained to it. The NS is used just to assemble the TS, but it does not bring anything new. You can walk and wander like this for a very long time :)

And then you start trying to understand what I've done: analysing features, target features, brute force, tuning and so on, with exactly the same success. Because the NS was again not used for its intended purpose.

It's easier to do random semiling of deals and signs, nothing will change, but the search will be faster. For some reason I realised this from the very beginning. Ah, because I read the books and it says the same thing :)

Until you replace the floundering with a normal teacher, it will be like this. A normal teacher is one who already teaches prepared patterns.

And for that you don't have the knowledge and agility. So the obvious option for beginners is to take prepared ts/signals and use them as a teacher.
 
Maxim Dmitrievsky #:
I don't do scrupulous preprocessing ever, no enthusiasm for it :) if the features are external, you need to choose them carefully. If they are derivatives of the original series, I don't see the point, just add a few variants.

I've already written above why all this doesn't work for fin BP. There the alpha is clogged with other uninformative examples from the rest of BP. You get memorisation rather than generalisation. And, on the contrary, strong regularisation also destroys TC, because not only uninformative but also good examples are extinguished, indiscriminately.

Cleaned up the redundancy. Long text is usually either desecrated or not understood by local weirdos :)

Your experience confirms your words, my experience confirms my words.

There are contradictions, but instead of arguing on this issue, isn't it more sensible to join the effort and benefit by enriching yourself with the achievements of the opposing side?

 
Aleksey Vyazmikin #:

Your experience backs up your words, my experience backs up my words.

There are contradictions, but instead of arguing on this issue, isn't it more sensible to join forces and benefit by enriching ourselves with the achievements of the opposing side?

Not more reasonable, I don't need help. The forum is even more of a distraction than it is a clue. Just stating my experience in general terms. Sometimes I sell it :) whoever hears it will save time by inheriting it.


Overshooting signs is a flawed tactic very ineffective, this is already as an axiom for me. I'd like to say IMHO, but it's more like an axiom.

 
Maxim Dmitrievsky #:

Overshooting signs is a flawed tactic very ineffective, it's already like an axiom for me. I'd like to say IMHO, but it's more like an axiom.

Now we're down to target selection, and we're getting to RL.

 
mytarmailS #:

We are left with overshooting the targets, we are slowly getting to RL

Passed it already :) the weak point is the approximator, which can't isolate and then generalise. But it is useful for general education. Does not distinguish NS between quasi-useful and actually useful. Because of the small fraction of actually useful in the data. Adding a little bit of useful to the data would work.

There's either follow the more effective innovations and be one of the first to apply before it becomes mainstream, or your own bicycling.

Transformers didn't work, let's see what else they come up with.
1...300030013002300330043005300630073008300930103011301230133014...3512
New comment
Reason: