Discussion of article "Creating Non-Lagging Digital Filters"

[MetaQuotes]

New article Creating Non-Lagging Digital Filters has been published:

The article describes one of the approaches to determining a useful signal (trend) in stream data. Small filtering (smoothing) tests applied to market quotes demonstrate the potential for creating non-lagging digital filters (indicators) that are not redrawn on the last bars.

Cluster Filter

Cluster filter is a set of digital filters approximating the initial sequence. Cluster filters should not be confused with cluster indicators.

Cluster filters are convenient when analyzing non-stationary time series in real time, in other words, stream data. It means that these filters are of principal interest not for smoothing the already known time series values, but for getting the most probable smoothed values of the new data received in real time.

Unlike various decomposition methods or simply filters of desired frequency, cluster filters create a composition or a fan of probable values of initial series which are further analyzed for approximation of the initial sequence. The input sequence acts more as a reference than the target of the analysis. The main analysis concerns values calculated by a set of filters after processing the data received.

Figure 1. The diagram of a simple cluster filter

In the general case, every filter included in the cluster has its own individual characteristics and is not related to others in any way. These filters are sometimes customized for the analysis of a stationary time series of their own which describes individual properties of the initial non-stationary time series. In the simplest case, if the initial non-stationary series changes its parameters, the filters "switch" over. Thus, a cluster filter tracks real time changes in characteristics.

Author: Konstantin Gruzdev

[Sergey Pavlov]

Thanks to the author for an interesting article.

[Mikhael Isakov]

The idea of building a non-lagging filter has been occupying me for the last couple of years. I've also been getting regular results that seemed better than others, and even non-lagging. All of them broke down in a step analysis. I ask the author to realise the input model signal as a Heaviside function: a step, up to moment T level =0, after one. And show the filter line on this. I am 99,99% sure that we will see a curve going to level 1 for some time interval, that is lag. An ideal, true, non-lagging filter should be able to handle the "steps" so that the output is equal to the input. If this is not the case, reasoning about "non-latency" is tinkering. Although I support the author: creation of a non-lagging, non-drawing filter is possible.

P.S. Comparisons with other filters should be given on a "step", placing all filters on one graph.

P.P.S. If this is already done in some video - please excuse me, at the moment (from a working computer) I can't see the video (apparently, posted on youtube ?).

[Konstantin Gruzdev]

I will add a test with a step for Momentum with a filter.

The reaction of the filter to the steps in the quotes flow can be clearly seen in the fourth video.

[Mikhael Isakov]

Lizar:

I will add a test with a step for Momentum with a filter.

What is the reaction of the filter to the steps in the quotes flow can be clearly seen in the fourth video.

Let's say that the period in some effective (not literal) sense of averaging is 101 bars, hence the lag is 50 bars. And on a clean step I can see that very well. But on the quotes, where two horizontal sections (0 and 1 conventionally) taken together are only 20 bars, and around - quotes - there I will not see anything like that.

The filters should be investigated by inputting ONLY model signals, and after that, after understanding their nature, enjoy the pictures of work on the quotes.

[Konstantin Gruzdev]

MAIS:

Let's say the period in some effective (not literal) sense of averaging is 101 bars, hence the lag is 50 bars. And on a clean step I can see that very well. But on the quotes, where two horizontal sections (0 and 1 conditionally) taken together only 20 bars, and around - quotes - there I will not see anything like that.

The filters should be investigated by inputting ONLY model signals, and after that, after understanding their nature, enjoy the pictures of work on the quotes.

I understand, I will add such a test.

[Mikhael Isakov]

Lizar:
I got it, I'll add such a test.

Thank you. I'm guessing what we'll see. Maybe your filter will show less lag with better smoothing than the competition. But it will 100% not be a lag-free, non-drawing filter. The next question - is your filter close enough to a non-delayed non-drawing filter to be used in this role? The test is simple, imho. If the filter really did not lag and did not redraw, it is reasonable to assume that a simple trading strategy would be profitable: buy if the filter is above the price, sell if it is vice versa. We will implement it by entering in each bar (or if the divergence is more than a certain threshold), and enter with the same lot and equal TP=SL. For example, TP=SL=50 pips. The point is that if we enter randomly, we have 50% probability to catch SL, without taking into account the spread. Taking into account the spread - more. And that's why we slowly lose money. If the filter does not really lag, the probability to catch a TP is higher (without taking into account the spread). If there is an effect, we cannot fail to notice it after analysing a rather long piece of quotes. If you show that the probability of TP in such a simple strategy is at least 55% without taking the spread into account, it is enough to compensate for the spread, and even earn. If your filter is really non-lagging, then by selecting the threshold of the difference between the filter and the price, when the module reaches the threshold, you will be able to show the probability of TP with the spread of 70, or 80, or 90+ per cent.... If yes - your filter is almost non-delaying and non-drawing, and your trading strategy is the Grail. If not - and the probability of TP is no more than that for SMA - sorry.

[Konstantin Gruzdev]

MAIS:
Thank you. I'm guessing what we'll see. Maybe your filter will show less lag with better smoothing than the competition. But it will 100% not be a non-lagging, non-drawing filter....

Let's wait for the publication of GMomentum_test indicator to draw conclusions not on guesses. It was supposed to be published together with the article, but apparently they did not have time to prepare it. Now it seems to be published on Monday.

As for tests in general, tests for linear filters are not suitable for cluster filters. More precisely, they can be done, but they are not indicative, because cluster filters are nonlinear. An example of this is the situation with a single pulse from the article. Most likely, a test with a single step function would turn out to be just as spectacular. But it will be, just as with the single pulse, just an amusing special case of smoothing. Nothing more.

[Mikhael Isakov]

Lizar:

Let's wait for the publication of GMomentum_test indicator, so that we can draw conclusions not on guesses. It was supposed to be published together with the article, but apparently they did not have time to prepare it. Now it seems to be published on Monday.

As for tests in general, tests for linear filters are not suitable for cluster filters. More precisely, they can be done, but they are not indicative, because cluster filters are nonlinear. An example of this is the situation with a single pulse from the article. Most likely, a test with a single step function would turn out to be just as spectacular. But it will be, just as with the single pulse, just an amusing special case of smoothing. Nothing more.

This is not about the filter test. It is clear that a non-delayed non-drawing filter, if it exists, is nonlinear. It is about testing a trading system in the geometry proposed by me, a priori profitable for THIS non-delayed non-drawing filter, and POSSIBLY profitable for some approximation to this ideal filter. The test with a step is not a special case, but a demonstrator of internal, innate qualities of the filter, which will always be present on any data, just on the step they are visible.

[Konstantin Gruzdev]

MAIS:

Here's what I got:

The grey dotted line is Heaviside, the blue thick line is Momentum, and the red line is Momentum with the filter on

[Mikhael Isakov]

Lizar:

Here's what I got:

The grey dotted line is Heaviside, the blue thick line is Momentum, and the red line is Momentum with the filter on

The grey dotted line is not Heaviside. The step should be vertical. If it's because the discretisation is so on the abscissa, and it's drawn with a line, it's a pity. And I meant to see at higher smoothing values. So that the lag would be greater than the 3 bars shown. And in general, if the lag is small, I usually practice repeating it as many times as necessary: I apply a filter to the result of filtering, after a thousand repetitions everything is visible, it becomes clear what fraction of the lag bar is contained in the algorithm at the apparent tiny smoothing.