Discussion of article "Brute force approach to pattern search"

 

New article Brute force approach to pattern search has been published:

In this article, we will search for market patterns, create Expert Advisors based on the identified patterns, and check how long these patterns remain valid, if they ever retain their validity.

A neural network is essentially also a kind of brute force. But its algorithms are very different from simple brute force algorithms. I will not provide the details of specific neural network architectures and their elements, but will try to provide a general description. I think, if we stick to a certain architecture, we limit the capabilities of our algorithm in advance. A fixed architecture is an irreparable limitation. A neural network is a kind of architecture of a possible strategy in our case. As a result, the configuration of a neural network always corresponds to a certain file with a network map. This always points to a collection of certain units. It is like with a 3D printer: Set item parameters and the printer will produce it. Thus, a neural network is a general code that does not make sense without a map. This is like taking any advanced programming language and simply creating an empty project without utilizing all its capabilities. As a result, the empty template does nothing. The same is with the neural network. Unlike brute force, a neural network can provide almost unlimited variability in strategies, any number of criteria and higher efficiency. The only disadvantage of this approach is that the efficiency greatly depends on the code quality. An increasing system complexity may lead to increased resource intensiveness of the program. As a result, our strategy is converted into a network map, which is its equivalent. The same is done in the brute force approach, but here we work with a simple sequence of some numbers. This sequence is much simpler than a network map, it is easier to compute, but it also has a limit in terms of efficiency. The below scheme displays the above explanation.


Author: Evgeniy Ilin

 
Interesting thoughts! But I would like to note that the sequence is chosen incorrectly, as it seems to me, quantised displacements do not reflect the geometry of the motion, and you do not do recovery in the end from the point of view of probability theory on an infinite sample you will get an infinite number of sequences of bar sizes, which sometimes give results. But in this way you cannot identify figures (patterns).... i.e. you can, but you will need a reconstruction operation.
Try to test your ideas with pure prices or normalised relative to the average with a large period (to exclude coefficient shifts due to the presence of a constant component), the main thing is to use this normalisation when working with the EA.
And Fourier series you threw in the furnace in vain, at the dawn of my youth I passed such a thing - cepstral analysis, so it allows you to get similarity coefficients for the same form of signals differing in amplitude and period, that the most interesting thing with a good approach analysis was realised on the SCVM planet3 (almost not a tube computer) in real time.
 
Interesting ideas and a competent approach.
It feels like something is missing to make your code start earning....
 
Aleksandr Martynov:
Interesting thoughts! But I would like to note that the sequence is chosen incorrectly, as it seems to me, quantised displacements do not reflect the geometry of the motion, and you do not do recovery in the end from the point of view of probability theory on an infinite sample you will get an infinite number of sequences of bar sizes, which sometimes give results. But in this way you cannot identify figures (patterns).... i.e. you can, but you will need a reconstruction operation.
Try to test your ideas with pure prices or normalised relative to the average with a large period (to exclude coefficient shifts due to the presence of a constant component), the main thing is to use this normalisation when working with the EA.
And Fourier series you threw in the furnace in vain, at the dawn of my youth I passed such a thing - cepstral analysis, so it allows you to get similarity coefficients for the same form of signals differing in amplitude and period, that the most interesting thing with a good approach analysis was realised on the SCVM planet3 (almost not a tube computer) in real time.

Fourier series may be and I will try in the future, in fact the regularities are still as distinguished, just not everywhere and at different periods differently. Here I was just trying to analyse approximately how long they work. In fact, such a deeper and more versatile analysis is impossible within one article. Calculations require a lot of power and time. And the programme is not the final version, I will add more functionality. So far, superficially. About bar shifts, you can add High and Low values in a row, and the average price, I thought about it, maybe I will make a modification soon. But I don't think it will be much better. The result depends more on the degree of freedom of the formula. And I wouldn't expect it to be anything super cool. The ultimate goal is simply to increase the quality of an analysis as the time of one analysis increases, and the system copes with this task. And it even finds regularities throughout the history. I just didn't show it )

 
Ivan Zaidenberg:
Interesting ideas and a competent approach.
It feels like something is missing to make your code start earning....

This code can earn, just need tests on demo at least, on different periods, with invert, without invert. Time is very limited so far. And in general, it is better to earn anywhere but at Forex )))) broker yes earns )).

 

1) Reminds me of Ivakhnenko's MSUA

2) The idea that any sequence of numbers is given by some algorithm is not true. From the theory of algorithms we know that almost all sequences have no algorithm. Accordingly, any complex and tricky algorithm will sooner or later lead to errors. Therefore, "adult" financial mathematics uses mainly probabilistic models.

 
Aleksey Nikolayev:

1) Reminiscent of MSUA Ivakhnenko

2) The idea that any sequence of numbers is given by some algorithm is not true. From the theory of algorithms we know that almost all sequences have no algorithm. Accordingly, any complex and tricky algorithm will sooner or later lead to errors. Therefore, "adult" financial mathematics uses mainly probabilistic models.

The idea is exploited that a particular existing sequence can be described by some formula from a previously known family and then follows the bruteforce....

the same "optimisation by an optimiser" view from the side :-)

 
Evgeniy Ilin:

I may try Fourier series in the future, in fact, the regularities are still highlighted, just not everywhere and at different periods differently. Here I was just trying to roughly analyse how long they work. In fact, such a deeper and more versatile analysis is impossible within one article. Calculations require a lot of power and time. And the programme is not the final version, I will add more functionality. So far, superficially. About bar shifts, you can add High and Low values in a row, and the average price, I thought about it, maybe I will make a modification soon. But I don't think it will be much better. The result depends more on the degree of freedom of the formula. And I wouldn't expect it to be anything super cool. The ultimate goal is simply to increase the quality of an analysis as the time of one analysis increases, and the system copes with this task. And it even finds regularities throughout the history. I just didn't show it )

I will not argue with you, my primary higher education is essentially tied to the processing of sequential data (I can not disclose all the subtleties - now it is followed), and therefore, I simply advised you to get away from your strategy to pick a combination of "three - seven - ace" and turn to the figures that have a place to work always (at least with a higher probability), they have only 2 common disadvantages with your method - the floating range of price and the period of signal formation. Even in your approach it is essentially necessary to adjust the working TF and volatility to the calculated frequent market patterns (figures) - and the grail is in your pocket. And for this purpose your method will work, since it is so fast, then recalculate your formula on say 5 consecutive TFs (for example 1, 2, 3, 4, 5 minutes) and say 3 ranges of volatility (normalise volatility through simple coefficients 1, 0.8, 0.5) and if in some variant you will have a maximum function, it means that at the moment the market has such a ratio of period and volatility. Alas, this again does not guarantee that this ratio will last at least one more calculation period. Therefore, I recommended to pay attention to the cepstral analysis, where you will need to determine only 1 parameter - the threshold of compliance with the reference model, and the maximum length of the formation of this model in bars.... Well, technical analysis theorists have drawn a lot of models.

 
Maxim Kuznetsov:

exploits the idea that a particular existing sequence can be described by some formula from a known family and then follows the bruteforce....

Well, yes, only this idea is supplemented by the idea that inevitable errors of this formula can be taken into account by another formula. But inevitably there will be errors again, which we will try to take into account by the third formula - and so on ad infinitum.

In the probabilistic approach (regression, for example), it is assumed that the error is inevitable, but it is "well" organised (stationary process).

 
Aleksey Nikolayev:

Well, yes, only this idea is supplemented by the idea that inevitable errors of this formula can be taken into account by another formula. But inevitably again there will be errors, which we will try to take into account by the third formula - and so on ad infinitum.

In the probabilistic approach (regression, for example), it is assumed that the error is inevitable, but it is "well" organised (stationary process).

Unfortunately regression is well applicable to physical processes, i.e. to those matters where there are clear patterns or at least a strong enough moment of inertia, in the world of money, there is only a probability that the price will move in some direction - and according to my estimates for simple traders for some strategies this probability coincides very closely with 50/50 and it seems to be no problem, but this is on a very large period, and on a short period 15 blacks in a row and then "Zero" !!!!

As a result, theoretically it seems that everyone is a millionaire, but in practice the drain after drain....

 
Maxim Kuznetsov:

exploits the idea that a particular existing sequence can be described by some formula from a known family and then follows the bruteforce....

the same "optimisation by an optimiser" view from the side :-)

Everything is optimisation unless you exploit the physics of the market, any system we test first in one area and then in another. I don't use optimisation at all for that matter. Just fitting it to the site. Here too, just the idea is that this fitting takes place in several stages plus a bunch of filters that keep only the best variants. It's basically an automatic search for patterns in the selected plot. The prettier the variant, the more likely it is to be a variant and not a fluke.