Stochastic resonance - page 26
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
So the task has caught on. That's good. I will now try to express my thoughts. If I wrote something wrong (or not understandable - tell me).
grasn
- Yes, there really is an edge effect. But that's the point, it's often considered bad (interfering). What if this is where "stochastic resonance" manifests itself. Suppose this variant - the direction of price movement coincides with the direction of the curve (resonance), but it doesn't coincide (no resonance). Who checked it? (Maybe that's where the Grail lies). The various Henning, Hemming, Blackman, etc. windows cannot be discounted either. (reducing this effect).
- Regarding noise, WE always have a mixture of signal+noise. And we have no mechanism to separate it from the signal, mechanically (as in my example, closing the receiver and measuring its intensity). So I suggest another option. Start from the concept of energy. The energy of the signal moves the market. The noise energy prevents us from seeing (isolate a useful signal).
- How to act
...What do you mean by catching it? We are discussing it for twenty pages, if you read it carefully (take your time). Good overview of signal processing, thanks, but I recommend to dump it all, if you want to make a fruitful work on this subject, and replace it with normal digital low pass filter of FIR or IIR class, unless you are sure that low pass filter will find a true signal.
Regarding noise it would be better to select EVERYTHING instead of WE, it's more logical, but the edge effect is badly removed with this approach, it's a fact, and therefore you will not select any noise at the edges, the signal itself will be noise.
PS: As a result you will invent much worse filter than FATL. Don't fool yourself and me, making an adaptive filter is a whole art and the principles of its design are quite different.
Candid, this work describes the prediction of the price movement in terms of potential levels, you might be interested. The quality is not good, but it's a mono read, moreover there are only four pages of formulas and charts.
Hmm, didn't I gave a link to it in the beginning? :) As they say let the bread flow and it will come back to you :). Good job, imho.
- building an indicator
I remember when I was trying klot's library I made some kind of indicator. Only it should be run in visualizer and wait until there will be enough bars for FFT. Jumps there are clear from what, number of frequencies plays, if to increase sampling they will be smoothed. Of course klot library must be in place (there is one in CodeBase).
Hm, wasn't that the link I gave at the beginning? :) Like they say, put bread on the water and it'll come back to you :). Good job, imho.
Yeah, I'm screwed, at least sign where that from where and from whose feed swiped, Vot t remember after reading that you like the article, and who gave the link and forgot, yeah, remember the fish in Strugatsky said, "My years are not the same, wood saw...". But every cloud has a silver lining, people might download this doc one more time.
Attention all interested parties and viewers.
Here is the solution to the problem that I addressed to all of you a few days ago. In principle, there is nothing new in it. I simply implemented the program, which I described in the last post on this subject. The point of putting it out there is simple: I see several applications of this method, so it may be useful for someone. I also want to "demonstrate" the usefulness of the theoretical approach.
So, the problem is simple. There is a series {X} of random numbers obeying a certain statistic. The statistics is non-Gaussian because possible values of X belong to the interval [0,∞]. Point X = 0, in general, may not belong to the general population. The number of members of the series {X} is N, which is large enough to give some credence to the distribution based on the available data and other statistical parameters:
µ=M(X) is the mean value of the series {X}
D=M(X*X) - the variance of the series {X}
σ =√D - skew of the series {X}
Since the available series is bounded, all its elements belong to the finite interval [Xmin,Xmax], Xmin>=0.
We construct a moving average Y with the period M on the available series {X}. The averaging method can be arbitrary. As the result we get a new series {Y}, the number of members of which is obviously N-M+1. The set of series {Y} also belongs to a finite interval. Denote it by [Ymin,Ymax].
The question is how to calculate Ymin and Ymax from the statistics and the parameters of the series {Y}? I will write at the end what this may be used for.
The first step is to construct an analytic distribution of the series {X}. In Bulashev's book, I found only one distribution function, which has as its definition area [0,∞] the lognormal distribution. I won't say anything bad about it, but it didn't suit me.
Since the statistics of my (and many other) series is such that the probability density p(X) at X→0 and X→∞ tends to 0, I assumed the following general form for p(X):
p(X)=A*(X^a)*exp(-B*(X^b)), where a>0 and b>0
Accordingly, the integral distribution function is defined as follows: F(X)= ∫ p(ξ) dξ. Here and further integration limits from 0 to X are implied. Unfortunately, local editor doesn't let upper and lower indices on the site. One has to twist it. It looks messy, of course, but it can't be helped. ξ is just an integration variable.
In order to be able to do anything with it, this integral has to be taken in analytic form. By integrating over the parts and using the limit value p(0)=0, one can see that
∫ (ξ^a)*exp(-B*(ξ^b)) dξ = -1/(B*b) * (X^(a-b+1))*exp(-B*(X^b)) + (a-b+1)/(B*b) *∫ (ξ^(a-b))*exp(-B*(ξ^b)) dξ
That is, the index a of the value X each time decreases by b. If after k steps this exponent becomes equal to b-1, the integral is reduced to the tabular one. Hence, we can formulate the integrality condition explicitly:
a - k*b = b - 1, or a = (k+1)*b - 1, where k>0 is an integer.
However, since we still have to calculate the mean and variance, this integrability is not enough. Let us see what is required to compute explicitly all the central moments of this distribution. Obviously µ = ∫X*p(X) dX (here integrating to ∞). Let us compute µ as a function of µ(X), assuming in the integral the upper limit is variable.
µ(X) = ∫ ξ ξ*A*(ξ^a)*exp(-B*(ξ^b)) dξ = ∫ A*(ξ^(a+1))*exp(-B*(ξ^b)) dξ
That is, it is an integral of the same kind with exponent a1=a+1. For integrability, a1 must satisfy the same condition:
a1 = (k1+1)*b - 1, where k1>0 is an integer.
Comparing this with the condition for a, we get: b = 1/( k1 - k). Denoting n = k1 - k we finally get an admissible form of the parameter b: b = 1/n, where n>0 is an integer. Note also that the relation 0<n<=k should be satisfied.
Keeping all this in mind, we can obtain in explicit form not only the integral distribution function F(X), but all the central moments of the distribution as well:
F(X) = 1 - exp(-Z)*∑ (Z^i)/i!
Ml(X) = (k+n*l)!/(k!*(B^(n*l)) *{ 1 - exp(-Z)*∑ (Z^i)/i! }, where Z = B*(X^(1/n)) .
The constant A appearing in the function p(X) is calculated from the normalization condition and accounted for in these expressions. The summation sign ∑ in the upper line implies summation over index i from 0 to k, and in the lower line - over i from 0 to k+n*l . The value Ml is the l-th central momentum (do not confuse l and 1).
Note that all obtained functions turn to 0 when X=0, and have the following bounds when X→∞:
F(X→∞) = 1 (normalisation condition) and Ml(X→∞) = (k+n*l)!/(k!*(B^(n*l))).
Thus we obtain:
µ = M(X) = M1(X) = (k+n)!/(k!*(B^n))
D = M(X*X) = M2(X) = (k+2*n)!/(k!*(B^(2*n))
Now that everything is there for good, we can return to the original series. The resulting final distribution function p(X) contains three parameters which can be used to ensure that p(X) best reproduces the statistics of the series {X} - B, k, n.
You could, of course, find them by MNC, but that's boring. I have made it simpler for my series. From the above formulas it can be seen that
D/µ^2 = (k+2*n)!*k!/((k+n)!)^2
Thus, the value of D/µ^2 is independent of B. Since D and µ are known for series {X}, we just need to choose such a pair (n,k) that would give the closest value. I just built a table on possible values of (n,k) pairs, and found only 4 suitable ones: (2, 3), (3,8), (4,16) and (5,26). The value of B is now determined elementary from expressions for D or µ.
Interestingly, the values (n,k) of the first two pairs (I did not check the others) gave excellent reproducibility of the experimental distribution curve p(X). At least for me this quality is excellent.
An interesting question occurred to me along the way. May be someone can enlighten me why such a simple and convenient distribution function with good properties is not used in statistics? And if it is used, why isn't it written about? I've never seen anyone try to approximate an incremental distribution other than the lognormal.
The third stage of the Marlesonian ballet involves the calculation of some limits X1 and X2.
The construction of the series Y = ∑ X is associated with the averaging of M values of X. It is reasonable to suppose that Ymin (a theoretical minimum) can be obtained if M of the smallest values of X fall within the average.
On the OX axis, M of the smallest values of value X occupy the interval [0, X1] and M of the largest values of value X occupy the interval [X2, ∞]. This is actually the definition of the values X1 and X2 .
Since there are N elements in the series {X} in total, F(X1) = M/N and 1 - F(X2) = M/N .
The function F(X) is known in analytic form, therefore the above equations for determination of X1 and X2 are analytic, albeit transcendental, equations. Any numerical iteration method can be applied to solve them. Since, as can be seen from the graph below, the function F(X) is monotonic, one can quickly arrive at the values of F(X1) and F(X2) starting from the inflection point using the gradient descent method. When calculated with the maximum accuracy allowed by MQL, 13-14 steps and less than a second of time were sufficient to obtain the values of X1 and X2 . The time was practically the same for pairs (2,3) and (3,8). Still, MQL is a good thing. (What a matcad .... J)
I hope it is clear where p(X) and where F(X) is.
Fig.1.
It would also be interesting to look at the dependence of X1 and X2 on M, or rather on the M/N ratio. But we will put it aside for the moment as there is not much time left. Note only that in the limit, when M→N, X1→∞ and X2→0 must hold. And we will deal with the definition of the final goal of this whole story, the values Ymin and Ymax.
In fact now it is very simple. The interval [0, X1] gives the position of M least X and [X2, ∞] gives the position of M most X. Our task is to determine the two averaged values on them. If the averaging algorithm is not trivial, then the problem must be solved separately for each particular case. If it corresponds to a simple MA, we can use the formulas:
Ymin = M(X1)/F(X1) and Ymax = (µ - M(X2))/(1 - F(X2)).
These formulas have a simple "physical meaning", so I do not go into an explanation. Instead I will present the graph of dependences Ymin and Ymax on Х1 and Х2 values. It shows Ymin in red and Ymax in blue. The horizontal line in turquoise indicates the value of µ.
As one would expect, Ymin at X1→∞ and Ymax at X2→0 both tend towards µ, one from below and one from above.
Figure 2.
Both of them correspond to M→N, what is perfectly clear from the graph of dependence of X1 and X2 on the value of M. Didn't I already gave it? Yes, you did. This is the very first of the graphs. And of the two curves you should use the curve F(X). But you should not determine F by X, but vice versa, you should determine X by F. While doing this, you also have to look at the equations for X1 and X2 and remember that if M→N, then M/N→1.
Thus it turns out that when M/N increases with M, X1 increases (and Ymin increases with it) and X2 decreases (Ymax decreases with it). But it is always Ymin< X1 and Ymax>X2 .
In my calculations I obtained that 1 - 3 - 5 values of series {X}, depending on the value of N, can go beyond the upper bound of X2 (the lower bound in this sense is of no interest). At the same time the value of Ymax is never exceeded. Which is, in general, understandable: the case where all M values of X are greatest is exceptional. As for the values of series {Y}, the probability of them exceeding X2 is even smaller. Not to mention Ymax.
So we have two estimates of the range of {Y} values, a hard one and a soft one. We can use either of them, depending on the requirements of the problem.
PS
Sorry. I can't insert pictures. Not in any format. The site must be glitchy.
And finally, why all this is needed.
While I was at it, I saw several possibilities for using it all.
1. Normalization of all known TA indicators, in particular - oscillators. Has anyone paid attention to the fact that oscillators can be used only in a narrow range of their smoothing periods? As the period decreases it starts to toss back and forth, and as it increases the amplitude decreases so much that it fails to reach the levels. Example of a favourite RSI below. Two variations for the two periods 14 and 30. If you rely on the second, you may not trade at all. The 70/30 levels are very rarely reached. Or these levels should be optimized anew for each period.
Fig.3.
TA indicators do not practically depend on t/f, as far as I understand it, this is a peculiarity of their statistics. But here, if the smoothing problem were solved, maybe something new could be gained from them. With such a stochastic normalisation procedure I think this is quite possible.
2. My personal problem was that the spread of the series depends significantly on N. How else could it be, did Hurst suffer for nothing ? :-))
Now I can bring everything to a universal standard, in which neither switching to another t/f, nor changing the smoothing period will affect the range of the series values. This allows the same levels to be used for input/output for all other parameter values. Optimisation makes sense under such conditions. By optimising on one t/f, I can then check the profitability of the strategy on another. If it persists, it means that the strategy really works. If it does not, it is discarded.
Maybe it will be useful to someone else.
3. So far nobody has managed to normalize the price chart directly. But it would be nice to do so. We are interested not in the absolute value but in its fluctuations. Perhaps we will manage to do it that way. Those who wish may try it.
4. In neural networks which I know nothing about, it is necessary to normalize data. Going beyond the boundaries of the conditioned range leads to loss of neuromouse.
Perhaps this way of normalisation will prove to be more useful in some cases than what is currently used.
That is all. Criticism is accepted in any form.
PS
I intentionally did not post any code or code samples. The algorithm is not described in detail, but very detailed. It is easy to understand. If you want to, of course.
I encourage the community to follow my example.
The reasons are as follows.
This cake is not ready for consumption. It is not a final solution, but a method. Any use of this method in private tasks will still leave them as private tasks. Those who, without bothering to understand and meaningfully use it for themselves, will rush to use other people's solutions, will be misled, will have wasted time and possibly money.
In order to use this method correctly, you need to
1. Formulate what your series {X} is.
2. Correctly form it with an appropriate procedure.
Examine its statistics, calculate statistical parameters.
Explore the correspondence of statistics of this series on different t/fs.
5. Find the appropriate statistical pair k,n.
6. Calculate B parameter.
7. Construct a model distribution function p(X) and compare it with the experimental one. Further use of this method will only be correct if the fit of the model and the experiment is satisfactory. And for this it is necessary to have an estimation criterion.
8. And finally, it is also necessary to be able to correctly use the obtained Ymin and Ymax. It's not as easy as it may seem. :-)
So, fellow programmers, do not only avoid encouraging freebies, but also give others a chance, an opportunity to show initiative and figure something out for themselves.
A programmer is not the one who programs everything he can get his hands on.
Just like a man is not someone who drinks everything that burns and eats... everything that moves.
I do, of course, but it's called 'through the back door'. I hope it's a temporary measure. As soon as it's working properly, I'll put the pictures in where they belong.
PS
Alas, even that doesn't catch on.
Moderators, HOOOOOOOOOOOOOOOOOOO!!! Fix the site, pls. No picture to attach, no file ...
to Yurixx
Amazing, instead of just looking at the spread of the resulting moving average series, we have quietly peacefully reached a theoretical estimate, and by gradient descent, based on an unproven distribution. That's theoretically cool!
Okay, I do not believe it yet, after a business trip to the glorious city of Kurgan, I will reread it again. :о)))
PS: Remembered a case from my near-scientific life. Came to my boss with a long roll of derived formulas, after looking it up, he said that there were no mistakes, but it could be simpler. To this remark I proudly replied "we don't look for easy ways", to which he immediately replied "that's why you don't find it".
to Yurixx
Amazing, instead of just looking at the spread of the resulting moving average series, we have quietly peacefully reached a theoretical estimate, and by gradient descent, based on an unproven distribution. That's theoretically cool!
In my opinion, it is simpler than that. A small piece of code that calculates normalization coefficients depending on t/f and averaging parameters is embedded into indicator or advisor init(). It works. How can you do it in the Championship if the Expert Advisor is not on your computer and the history is loaded on the unknown volume?
But that is a trivial matter. I have a more serious question. Do you have to recalculate these ratios every time you change a symbol, t/f, etc., by hand or using Matkad :-)? ? Don't you get tired of it? Or create a database for all symbols, t/f, smoothing parameters etc ? ? :-)
There is one more, most important point. But if you haven't noticed it, whatever. :-)))
By the way, the "unproven distribution", given that in forex no distribution of any value is known (only that it is known to be non-normal) is ridiculous. It's a good joke.