Adaptive digital filters - page 8
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
2 Prival: I remembered that Kalman, according to you, is based on MNC. Now I see why it works great on radar data (with Gaussian distributed errors), but - worse on market data. The main reason Kalman is perfect on Gaussian data is that the error function (target) - the sum of squares of variance in this case - is perfect just for the Gaussian distribution. For other distributions, the error functions are different. For distributions with power tails (heavy tails), the target functions are quite different, and MNC does not count here. This is why JMA is better than Kalman on market series.
How interesting. Come on Alexey, I'll wave my birch tree over your head too :-). After all, 99% of disputes arise when 1 person says that it is better. But he didn't say in what sense it's better (where's the criterion + how much better). And let's say I argue that MA is even better, the best, just awesomely better :-). JMA and Kalman are not even close.
It's like saying one person is better than another. But not to say in what way (no criterion). Suppose the first is better at drinking vodka, while the second is better at shooting pioneers with a slingshot. The question is which of them is better ?
After all JMA is a black box to you and me. And the Kalman filter must contain
1. An observation model (signal model + noise model).
2. A measurement model (measurement error model).
And the solution to be found by ANC with a quadratic loss function. Based on a priori data and obtained measurements. And I want to note that such target error function works not only for Gauss, but for any symmetrical distribution law.
Mathemat and now a question. Suppose that the analyzed process is sinusoidal with uniform noise and measurements are subject to a non-stationary Poisson's law. All these models are nested in the Kalman filter and it has found the optimal estimate (of this wild mixture) by ANC (quadratic loss function) according to all nested a priori data when a new measurement arrives.
Where, in what place is the JMA black box better ?
If you put in a Kalman filter a model that is 100% consistent with forex (signal + noise) and an adequate measurement model (non-synchronous expert estimation system), it will be a hell of a machine.
Z.U. The observation and measurement noise model can be anything. The main thing is to be consistent with what is there.
Convinced, Prival. So there is something that Djuric is obviously not telling us, or is deliberately flaunting his product. And yet, and yet: why the OLS and not, say, the sum of the moduli of deviations? Simply because ANC is analytically more convenient?
P.S. I remember wondering about an expedient loss function in connection with my old research on neural networks (there it is a target function). And somehow either deduced or read somewhere that the sum of squares is directly related to the particular hypothesis about the law of error distribution (here - Gaussian). When in my research I changed the function to sum of moduli (i.e. changing a priori law of error distribution to exponential), the quality of prediction improved slightly, but not cardinally.
If you put a model in the Kalman filter that is 100% consistent with forex (signal + noise) and an adequate measurement model (non-synchronous expert judgement system), it will be a hell of a machine.
Z.U. The observation and measurement noise model can be anything. The main thing is to match what is there.
Does the model of the signal itself have any limitations?
to Northwind.
Thank you. And "we need a simple and sufficiently consistent concept of market life" - do you mean your own development or the use of some techniques, such as those described by Shiryaev?
...the sum of the squares is directly related to the particular hypothesis about the law of error distribution (here, Gaussian)...
Prival, Mathemat, I'm afraid to get annoyed again, but I have to say it again - there is virtually no noise in the quotes - that is the input signal. You are trying to use the tools of mathematical statistics (filtering is the same). Statistics of what? Statistics, laws of distribution, their moments of different orders refer to random variables (processes). If you get a tick, is that a signal or noise? I argue that it is a signal, because with this data you can give a buy or sell order, and it will be executed (all other general conditions being equal). Yes, it is difficult to predict what the next price value will be, so I want to believe there is a random component and a non-random one that can be detected and then extrapolated-predictable. And it's not random, it's just unknown. Or, if you like, all random - without dividing it into additive components. What are you going to separate? The same Kalman filter will filter out a very definite component - defined by your own model in the form of a smooth analytic function. Do you know it? I don't. You're trying to identify the dynamic properties of the market, and applying a physical analogy is, I'm afraid, also futile: you can find minute candles with an amplitude greater than a figure, as well as gaps, which indicates that it is practically inertia-free.
It is possible, accepting the hypothesis that price values are random, to investigate them using mathematical statistics. This is something Mathemat and others have long been fond of. The result is fat tails and, hence, again the lack of practical prospects.
But what about the positive results of the "pianists" and the leaders of the Championships! They are just the ones who speak of the need to expand the methodological paradigm. Elements of technical (and maybe even fundamental) analysis should be introduced into the MTS, but not directly, using the old "classical" recipes, but through the preliminary filtering of working models on the basis of the Bayesian approach. It is hard to cope with such amount of information "manually", but someone does it. The conclusion is obvious - to train a robot.
I have already made a probabilistic network in MQL but I cannot make it work with a profit factor higher than 1.5 - the teacher is too weak :-).
P.S. Another example to confirm the argument about absence of noise in price.
When they speak about measurement noise they mean a random deviation of measurement data from the true value of the quantity being measured. For example, the radar (for specialists :-)) gave a range value of 105, and the true value is 100, in the next measurement 99 instead of 101 and so on. The distribution of the error is generally normal. In case the price comes, for example, 1.2567 - this is its true value, the error is equal to zero! What kind of noise are we talking about?
Prival, Mathemat, I'm afraid to get annoyed again, but I have to say it again - there is virtually no noise in the quotes - that is the input.
No annoyance, rsi, normal discussion. By and large I agree with you: noise in quotes can only be seen within a certain interpretation, a model. When I talk about errors, I usually talk about prediction or approximation errors.
Prival is talking about errors of observation and measurement. This is quite natural in terms of his speciality. But these are very different errors. Nevertheless, this point of view has a right to life, although in my opinion it is artificial. Prival, no offence, but how you practically intend to implement your 100 MHz sampling rate, I have no idea yet.
I see the application of statistical methods to the processing of financial series as useful only in the context of assessing possible risks, no more than that.
Yeah, and there's also a five-minute candlestick on the oire of 198 pips in 2000. who's bigger?
Prival, Mathemat, I'm afraid to get annoyed again, but I have to say it again - there is almost no noise in the quotes - that is the input signal.
Rsi, on the contrary, I am very happy that you have returned to the discussion. After all, you're talking sense, you make me think. I apologize for myself and for everyone else, if I have said (or have said) something wrong. I had a mathematician who gave me such commands (generals relax :-)), made me do push-ups :-). I'll see him alive, I'll hug him like a brother. And you for me a yellow trousers, ns in MT4, wizard. Just like klot.
About the noise, I think about it all the time myself. I did the following. I took weekly quotes and started analyzing all components of this flow. First I subtracted the trend, then the fluctuations, everything I could get out. And after each procedure, looked at the residuals. When I selected all of them, there was noise in the residuals, but not Gaussian noise. Some strange noise +-1 pip and nothing else, some rare spikes of 2-5 pips and 1 gap was 40 pips (I was especially looking for a week with good gap). I sat down and thought about it and I think I found an explanation for this noise. Most probably it is a measurement noise, if you look at the quotes from the ADC point of view (it is quantization and sampling noise) they should be physically there if we digitize a continuous process. So I think you may be right that there is no noise there - it's a pure signal. But there is one nuance, which is bothering me :-(.
There's no way to make 100MHz it's not possible (otherwise I'd already be in the sweet spot :-)). The only way to improve this situation, at least a little, is to do as they do in a normal DC. We have to take the maximum number of suppliers of quotes and process this flow by ourselves (and not build a candle, but an ellipse of constant probability). After all, we must conclude transactions (Buy and Sell) in accordance with the data provided by brokerage companies. But we do not have to use only the data of this brokerage company to make a decision concerning Buy or Sell; we may not use the quotes at all :-)
For the amount of deviation modules. If I am not mistaken, with this approach the valuation is either biased or untenable. I don't remember exactly, I'm afraid of making a mistake, but something about the power of the estimate. Although you can choose a non-quadratic one. Basically any, the main thing is to determine in which direction from the middle (the ideal observer) errors are more important, let's say in one direction it is a square in the other cube. This is from the statistical theory of decision making. (Wald's book "Statistical Decisive Rules", I think it's in there). If anyone needs a book I can post it.
NorthernWind
I do not understand the limitations you're asking about, please clarify the question.
The model must be represented in the form of a system of stochastic differential equations. And the main condition is that it must be adequate to the process being filtered. These are exactly the kind of constraints we have.
Z.I. Mathemat wanted to offend me :-), you can't wait. Those whom I respect, they just can not do it. Unless he shakes my hand and throws the cognac away (so long and carefully preserved for him), and then I'll think first, maybe I did something wrong. And then I'll be the one to carry the water :-)
I agree that things are bad at the gap points and perhaps there is no inertia. And there is no way we can get rid of all gaps (I put forward a hypothesis that it is due to discrepancy of sampling rate to the analyzed process - does not seem to contradict anything). But gaps are not all 24 hours in a day. ACF analysis shows that the process is correlated, there is a correlation time and hence the process can be predicted. After all, it is almost a direct analogy with a physical process, an aeroplane cannot turn instantly, mass interferes, there is inertia, the process can be predicted because it is correlated. Without this and NS would not work IHMO (I have read here that this acronym can be interpreted in different ways, I have this in my humble opinion).
But what about the positive results of the 'pianists' and the Championship leaders! That is exactly what they say about the need to expand the methodological paradigm. Elements of technical (and maybe even fundamental) analysis should be introduced into the MTS, but not directly and with old "classical" recipes, but with a preliminary filtering of working models based on the Bayesian approach. It is hard to cope with such amount of information "manually", but someone does it. The conclusion is to train a robot.
The results are encouraging and keep my hands up (economists talking about martingales and Wiener process). I don't know if I'm following the old "classical" recipe (although I've read books about forex analysis, not all of them, but there are dozens of good ones). I searched as much as I could not find the results of applying Kalman filter to analysis of quotes. They are either absent or on the contrary, those who managed to implement it carefully conceal their results. For it is very important to select working models using Bayesian method. It will not work otherwise. I try to do it manually - it's hard, but it is interesting. I don't trust algorithms, where I don't know how decisions are made for me. I do not like them.
The Kalman filter very rarely manages to be implemented in real life. It can be regarded as an ideal, as the well-known Bayesian solution, when the most powerful estimate is chosen in the presence of a priori and a posteriori information.
It goes like this.
And me and Mathemat as well as some others have seen this noise on ticks. Moreover, on the ticks it is clear that +-1 points has higher probability of the reverse movement than its continuation. Unfortunately, this regularity is inside the spread. And it is not high.
But the fact that it appeared after processing is interesting.