Machine learning in trading: theory, models, practice and algo-trading - page 1539

 
Igor Makanu:

voice and music files are compressed with dtw, and they are also BP

;)

Igor Makanu:

I've already studied this topic, in my own words, it's something like this:

)) what the hell compression? what you studied there is not clear, but certainly not that

Maxim Dmitrievsky:

Why do you need to compress fin. rads :)

and really, why? :))

 
elibrarius:

You can simply discretize the input data, e.g. convert 5 digits to 4 digits. And the data will already be in groups of 10.
Or as I suggested before - build in alglib forest stopping branching when required depth or number of examples in sheet is reached.

Discretization works better than cat fiches+vanhot, it did not improve significantly

 
mytarmailS:

and really, why? :))

you can compress the quotes into zip archives, look at their size and they will be new features

 
mytarmailS:

)) what the hell compression? what you studied there is not clear, but definitely not that

Let's consider that you've glossed over, at least read Wiki before you write

the dtw algorithm analyzes the time component of the signal, and reduces it to a constant value. Knowing this value, you can simply remove pauses between the information components of the signal - as a result, you'll have data packets without time components + a constant algorithm for transforming the time axis


Maxim Dmitrievsky:

you can compress quotes into zip archives, look at their size and they will be new features

No way, zip algorithm is a strict transformation algorithm, and you can't identify data that differs by 1 byte as the same data,

you don't need a strict algorithm to work with data, they all have a loss of the original data, if you don't make it up - it's jpg - it compresses with a loss and data that is close in content will be restored approximately the same in the end, but it's visual! - the checksums will be different, the bytes themselves will have different values....

but as a training example for the NS, maybe that's what you need, i.e. a jpg for arbitrary data (not pictures)

 
Igor Makanu:

It won't work, the zip algorithm is a strict conversion algorithm, and you can't identify data that differ by 1 byte as the same data,

you don't need a strict algorithm to work with data, they all have a loss of the original data, if you don't make it up - it's jpg - it compresses with a loss and data that is close in content will be restored approximately the same in the end, but it's visual! - the checksums will be different, the bytes themselves will have different values....

but as a training example for the NS, maybe that's what we need, i.e. a jpg for arbitrary data (not pictures)

I was just kidding )) Well the autoencoder or convolution is good for this task. Vladimir has some articles about encoders, but not about convolutions.

seq2seq is also in fact a decoder-encoder. For example, in machine translation there are different numbers of letters between Russian and English words. It all is compressed there, analyzed and then uncompressed.

https://medium.com/@gautam.karmakar/summary-seq2seq-model-using-convolutional-neural-network-b1eb100fb4c4
Seq2Seq model using Convolutional Neural Network
Seq2Seq model using Convolutional Neural Network
  • Gautam Karmakar
  • medium.com
Seq2seq model maps variable input sequence to variable length output sequence using encoder -decoder that is typically implemented as RNN/LSTM model. But this paper https://arxiv.org/pdf/1705.03122.pdf shows application of convolutional neural network for seq2seq learning which is state of the art for computer vision using deep learning. There...
 
Maxim Dmitrievsky:

I started to read something about dtw, but I don't understand how to apply it to finance series and why I need it) but it's an interesting topic, I guess.

There was one concept where DTW was an extension of the process of finding semblance fragments of a plot, but in the end wavelets are probably easier (or maybe not).

 
Maxim Dmitrievsky:

That was a joke)) Well an autoencoder or convolution does a good job. Vladimir has some articles about encoders, but not about convolutions.

seq2seq is also essentially a decoder-encoder. For example, in machine translation there are different numbers of letters between Russian and English words. All this is compressed there, analyzed, and then uncompressed.

https://medium.com/@gautam.karmakar/summary-seq2seq-model-using-convolutional-neural-network-b1eb100fb4c4

I read about encoders last year, I think, that everything, as usual, will be limited by large studs on a price chart - they break any transformations, if there were no such studs, the MA bars would work, the Kalman filter would work too and everything would work fine ))))

If we look at the data lossy compression, we may try to do some search - the samples themselves will be compact in size, it means the network will not be large and usually it learns better


PS: yes there was a DTW on the forum, even searchhttps://www.mql5.com/ru/code/10755

twisted it once, but ... all should be treated with a file before use )))

 
transcendreamer:

There was one concept where DTW was an extension of the process of finding semblance fragments of a graph, but in the end probably wavelets are easier (or maybe not)

Igor Makanu:

PS: yes there was a DTW on the forum, even searchhttps://www.mql5.com/ru/code/10755

I twisted it once, but... everything should be filed before use )))

Oh no, the hell with it, I'll go on neural networks twist. I don't believe in such a thing.

 
Maxim Dmitrievsky:

Oh no, fuck it, I'm going to go back to neural networks. I do not believe in such a thing.

I don't want to do it either, I'm busy with MT5 tester and its GA, GA works quite adequately, you can quickly (4.5 x 10^142 variants tested!!! - about 2-3 hours) build automatic TS, and then test it on a forward, the results are quite acceptable, imho

HH: but GA also need a file to finalize, it is difficult to pick up the input parameters - give a lot will be a test of 5 years, you start cutting as not correct input data - here as luck or immediately found where to put and whether at all will not find

 
Igor Makanu:

I don't want to do NS either, I took MT5 tester and its GA, GA works quite adequately, you can quickly (4.5 x 10^142 variants tested!!! - about 2-3 hours) build automatic TS, and then test it on a forward, the results are quite acceptable, imho

I'll either do monitoring soon, or write an article... I'm too lazy to bother with the article, it's too much typing.

Everything is purely in python now. It saves a lot of time.

GA's flaw is that it does not generalize
Reason: