FR H-Volatility - page 6

 
Mathemat:
Yurixx wrote (a): Or maybe a Wiener ? :-)
The Wiener process is, by definition, the integral of the Gaussian white noise, while the return process is not Gaussian at all.


That's why there is a smiley face there. Until recently, there was a very active discussion about the measure of "randomness" of quotes flow. Wiener process was used as a benchmark and the quotes flow was compared to a Wiener one side or the other. The main point of discussion was not even about the form of the FR, but about whether the quote flow reflects some stable patterns (not necessarily deterministic) which can be exploited, or whether it is still arbitrage-free SV, hoas etc.

What is your opinion now, Mathemat, how much do statistics rule in forex and what should we look for: statistical superiority (in the sense of imperfection, market arbitrage) or still a pattern ?

 
Statistics rules, of course, Yurixx, what else can it do with such nasty processes... A strict law is unlikely to be found, but statistical superiority... that's what all the system builders do, but almost always in a one-sided manner.

A reasonable researcher understands that a system capable of showing a modest profit factor of not less than 1.5 (with, say, 99% computable probability) steadily and provably, is not at all easy to make. In the overwhelming majority of cases that I see here some statistical conclusions about the system are made only based on the analysis of a trader's subjective view of the market (in terms of a particular system) - but I see very few conclusions related to the statanalysis of objective properties of market series. But these are two equal components of the system! The system may be great on the backtest, but where in it is the analysis of variability associated with different realizations of the time series!

By the way, I dare say there have been some interesting topics on the analysis of the second component recently. But these topics require deep and thorough work, rather than simply combining the parameters of yet another grail (with a small letter)...
 

What do you mean "sustainable and provable"? Proven theoretically? Or practically? Why is stability alone not enough?

скромный профит-фактор не менее 1.5 (с вычислимой вероятностью, скажем, 99%)

And how can we calculate the a priori probability of the profit factor? Again on the basis of a theoretical model? So it is supposed to be possible to build a model that will adequately reflect the nature of processes in the market. Then it will be possible to prove, to calculate and to make objective conclusions. But all this is utopia, and I think that you will not argue with me. Probably sometime science will be ready to offer such theory, but meanwhile we are at a level of Carl Linnaeus in biology: we classify candlesticks and patterns. :-) That's why I personally cannot say that statistics rules. I have not seen any TS, which is based on statistical properties of quote flow and which has a stable income. The first theoretical attempt is Pastukhov's thesis, for which I want to thank him very much. But there are profitable TS, and Better's Expert Advisor is not the first and not the only one. But so far they are all built on deterministic (and, alas, not theoretical) models. Even Better's PNN exploits no statistical properties of the series at all, and its probabilistic name tells only about the nature of the decision, not about the basis for that decision.

And I see very few conclusions related to the statanalysis of objective properties of market series.

This place in your post interested me very much. I've been getting into statistics lately. :-))

Of course, my investigations of quote flow statistics can hardly be called sensible, as I'm not an expert in this field. Nevertheless, they have added much understanding to me. Moreover, they have added to my desire to make them sufficiently deep, meaningful and constructive. So I want to ask you:

1. What is "statanalysis of objective properties of market series" ? What does it involve ?

2. Which "objective properties" should be investigated?

3. What conclusions about which statistical aspects of the market should (or can) be made in order to base a question about the construction of the TS on this basis ?

4. How do you see the construction of a TS based on knowledge of the statistical properties of market BPs ?

Just don't be intimidated by the straightforwardness of these questions. I realise that you, like everyone else, don't have any comprehensive answers. Those who have, are no longer among us. :-))) I put them in the hope that in the process of discussing them here, as has happened more than once, worthwhile thoughts may arise and progress may be made.

 
Yurixx писал (а): What do you meansteady and provable? Proven theoretically? Or practically? Why is sustainability alone not enough?

...

1. What is "statistical analysis of objective properties of market series"? What does it involve?

2. Which "objective properties" are to be investigated?

3. What conclusions about which statistical aspects of the market should (or can) be made in order to base a question about the construction of the TS on this basis ?

4. How do you see the construction of a TS based on knowledge of the statistical properties of market BPs ?

Yurixx, it's a long story, I've had such thoughts in my head for at least a year. I hoped that I will be able to develop it all myself into a technical article (or several articles) with codes in MQL4, but it's not working out - both for lack of time and for technical reasons.

The primary question was this: "Why do the vast majority of TSs whose signals are based on traditional trading interpretations of conventional indicators drain?". The usual answers - "the market is changing", "the quotes of different brokerage companies are different", "insufficient depth of testing or optimization", etc. - I am not satisfied with such answers, because this way we can "prove" the failure of any TS when it really happens. I am only satisfied with clear mathematical reasons for failure. The result of my analysis is the first of the alleged articles, which provides, in my opinion, a fairly convincing critique of the principles of TC construction.

Unfortunately, this criticism, while clear ideologically, is still not supported technically (code). The main reason for this is that I have not yet learned how to do the most important thing - to generate synthetic stories (bar, of course), about which I could say that they do not differ from real ones in their main characteristics.

What are these synthetics for? Simply to make the material on which the strategy is tested statistically representative. The main problem for the system builder is not enough data. Testing and optimisation is always done on a single 'realisation' of a historical process. No forward analysis, multi-currency and multi-period testing described in Pardo's classic book will still provide the necessary statistical confidence that the system will still work, and Pardo himself admits this. If we run the TS on a few thousand synthetics, we can draw quite confident conclusions about the robustness of the strategy (not on the parameters of the TS, but in the space of all synthetics!). But this requires high-quality synthetics. Then the testing process can be taken to a qualitatively new level. The very methodology of such "synthetic" testing is the subject of my proposed second article.

The lack of knowledge of the fundamental statistic characteristics of real financial series is the only serious obstacle that prevents implementing the idea in reality (it became clear after consulting with xeon and komposter). Everything else can be implemented in MQL4 with addition of several undocumented features of the language.

That's actually the answers to almost all your questions, Yurixx, i.e. an attempt to prove the robustness of TC. I won't answer question 4, because I don't suppose to use statistical properties of financial series in TS. Personally I need it only for qualitative testing.
And how can one calculate a priori the probability of a profit factor ? Again on the basis of a theoretical model? So it is supposed to be possible to build a model which adequately reflects the nature of market processes. Then it will be possible to prove, to calculate and to make objective conclusions. But all this is utopia, and I think that you won't argue with me.
So Yurixx, it's not utopia, but only under one condition - if I manage to calculate the stable characteristics of series (ideally find such their transformation so that they become stationary, and then complement what I obtained with the key hypothesis of ergodicity, which itself justifies such testing).

Anyway, I hope that the topic will be of interest. I want to warn beforehand: not all TCs are tested this way! Not those that exploit the discrete properties of the market (Fibo, channels, support/resistance levels). It is very likely that I will still publish the articles as they are now (philosophical and analytical), but they will be more of a manifesto type thing rather than a specific technical solution. There are other considerations there that I haven't mentioned here.

If everything written above is perceived as bred safe cable, just say so, I won't be offended.
 

On the contrary, everything is perceived quite clearly and positively. I see you don't give up on small problems either. There's a great place in "Monday" by Strugatsky. Two lines from the dialogue: "So this is a classically insoluble problem! - Of course it is. What is the point of solving solvable ones? You know in advance that they have a solution anyway" :-)) Very pertinent.

I'm not sure that the problem you posed has a solution. I'm referring to the generation of series that "do not differ from the real ones in their main characteristics". However, generating synthetic series that can simulate the behaviour of real series to a certain extent, and using them to test TC is a cool, fundamental problem.

My "physical" understanding of forex in a nutshell: forex is a stable, self-consistent system with large dissipation coefficient and small relaxation, which tends to its equilibrium. However, it is constantly perturbed from the outside and these perturbations are very diverse in strength, duration and nature. From weak, but constant and lasting, resulting in long trends, to very short, and without material basis, "psychological" impulses. To build a system that could reflect all of this is to build a fully working market model. That's what I think is impossible (at the moment), because in fact all the factors that affect the market in one way or another have not yet been studied at all. But to build a multi-parametric stochastic model that does not pretend to represent the market (i.e. generates series that are quite possible to distinguish from real ones) is quite possible and no less useful.

The task of TS is not to determine precisely where, when and how the price moves, but to dynamically extract from the flow of real data the nature of the influence that significant external factors have on the market, and adjust to that influence. If TS is able to do that, then it will not care whether the data is real or synthetic and it will react adequately. Thus, having a limited multi-parametric stochastic model, we can perturb its parameters in a random and/or deterministic way to get series, which are not real or close to them, but still can make the TS test reasonable.

So, Yurixx, this is not utopia, but only under one condition - if it is possible to calculate the stable characteristics of the series (ideally find such a transformation so that they become stationary, and then supplement what we have obtained with the key hypothesis of ergodicity, which itself substantiates such testing).

If I am at least somewhat right, there is no way to calculate the stable characteristics of the series, as they are constantly subject to perturbations, and this is all utopia. And we must enter from the other side, from the point of view that profit in Forex is obtained not from knowledge or use of statistics of series, but from knowledge and use of perturbation, which violates these statistics.

 
Yurixx писал (а): I'm not sure that the problem you posed has a solution. I am referring to the generation of series, which "do not differ from real ones in their main characteristics". However, the generation of synthetic series, which can simulate the behavior of real series with a certain degree, and using them to test the TS is a cool, fundamental problem.
Of course, only to a certain extent - say, the same way it happens with stationarity: there is in a broad sense (soft) and there is in a narrow sense (hard). It is virtually impossible to replicate absolutely all the parameters of a process. But if, say, it is established that its implementation is a stationary process in the broad sense (dependence of ACF only on the difference of the arguments, constant m.o. and FR), then, having created a process with the same parameters, we can say with some quite acceptable accuracy that the created process is indeed quite similar to the original one.

This problem is much more complicated than the trivial generation of the Wiener process, since in the Wiener one the neighbouring increments are independent, while in the FF series they are dependent. Nevertheless, legends about vinerability of financial series are stable and exist even in very serious trading forums. Of course, they are nourished by the external rough similarity of such synthetics and real series. Some even try to go further and find fibs, channels and resistance/support levels in Wiener series, thus "justifying" the validity of technical analysis. Well that, I'm sorry to say, is already a lunatic...
And we have to approach it from the other side, from the point of view that profit in Forex does not come from the knowledge or use of the statistics of the series, but from the knowledge and use of the perturbation that violates these statistics.
This is where our goals diverge, Yurixx. Your goal is to exploit perturbations to get benefits from them, while my goal is to find stable statistical laws of financial series that include these perturbations and serve as the basis for proper testing of our TS. In principle, this should not interfere with further studies of real financial series.
 
Mathemat:
That's where our goals diverge, Yurixx. Your goal is to exploit perturbations to take advantage of them, and mine is to find stable statistical laws of finanass series, which include these perturbations and serve as the basis for a qualitative testing of our TS. In principle, this should not interfere with further studies of real financial series.

Just listen to how cool it sounds: "sustainable statzakon finanasy rows incorporating these perturbations". :-)

If perturbations are by definition external to the market, if they are temporal (otherwise they would not be perturbations), then what kind of stability of laws, parameters and functions of the statistical process can we talk about?

Stability (neither in the strict sense nor even in the soft sense) will by definition not be fulfilled where one of the dominating factors in the process is replaced over time by another. For example, an economic upswing is replaced by an economic downswing. Or, as now, the financial system begins to crumble and undergoes collapse. I presume you are not going to find these sustainable statutory laws theoretically. If I am right, then tell me how in principle you are going to find these laws, where? By researching real historical data? How?

By the way, regarding my goal, - too loose interpretation. I didn't say anything about it, but at this stage our goals coincide - research into the nature and patterns of forex. And if they weren't, I'd hardly be having these 'clever' discussions. And I was only talking about winning as a measure of understanding of the processes we are dealing with. And if that understanding does come to pass, I will not deny myself the pleasure, as a reward, of drawing from that source. Neither will you.

 
Yurixx писал (а): How in principle are you going to find these laws, where? By researching real historical data? How?

Like Peters did in his book http://bigforex.biz/load/8-1-0-136. You could also read http://bigforex.biz/load/8-1-0-137, it's his first book. In my opinion, what you call perturbations fit his model quite well, as he does not single out these perturbations in any way, but simply looks at the process without cutting out the trend sections.

I know the task is very difficult, but the goal is very tempting. By the way, Peters is not the only one. There's also Shiryaev, who also studies financial series statistics.

And one more thing, Yurixx: I am able to make clever conversations, but in this case my goal is the most practical, just what young system-builders never do and do not consider it worthy of attention, i.e. an attempt to find an approach to prove the robustness of TS (statistical, of course). And it's a very good thing that such a practical goal should be based on such basic research.

There are not a lot of people decently versed in statistics here (I do not include myself to such), and, I am afraid, will have to look for mathematicians on the side. But there are people here who can code even the devil with MQL4, and that's very good.

 
Mathemat. I have a few questions, if you don't mind emailing or connecting at home on Skype.
 
Of course, I don't mind if you do. I don't always manage to Skype, because I get home late and don't want to wake up my family. But I can do it on Facebook, it's silent.
Reason: