Statistics as a way of looking into the future! - page 20
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
1. The price increment on the next bar.
I don't think the next bar is necessary. You need a prediction for a given time. That's why you don't take minutes.
How can I not? I only work with M1 (when there is no tick history).
As for the forecast horizon, it is defined by the value of time series. If it is a daily, then the forecast is made a day before, if it is a minute - then it is made 1 minute before. In other words, I don't think there's any justification for spending money on minute analysis for a day-ahead forecast. I think this empirical statement can be proven rigorously. The point is that forecast accuracy decreases exponentially with increasing forecast horizon measured in counts.
Well, I understand in general terms.
Piligrimm, the indicator you put together as you presented it, allows you to beat the market on H4 with statistical certainty. And that with minimal risk. What is the reason why you have not toppled the Forex market so far?
Well, what a question you are asking, however, really tricky. Now, I go to the grocery store for a bubble and I`ll knock it over and over Forex!
And seriously, it is simply impossible, even if all the private traders collude with each other and have the coolest MTS, even then their share in the market is so small and the buffer between them and the real market is so big that their efforts will not lead to anything. Even the cases that are supposedly known about, for example with Sores, are myths spread by DCs to entice customers. Sores was just at the right time in the right place in a big wave, created by English banks and made good money on it, and then these banks pinned all the blame on him to cover their machinations and mistakes, saying that he caused the crisis. Although I still believe that as the intellectual power of the systems grows, in 10 years the market will become totally different, but it will not be the merit of private traders, but the decision of those grey cardinals, who shape the market policy.
I will not assert anything yet, but I think that in this case the results were obtained incorrectly due to indicator re-rating on the zero bar. For a horizon of 1 bar, it is obvious - when emulating trades on presented data the decision is made at the beginning of the bar using data that in reality was obtained at the moment of its full formation. For a 2-bar horizon, we need to analyse in more detail". - The decision may be made at any time during the formation of the zero bar, and the system may be several times to close in different directions, while receiving a profit, but it can be seen only in the dynamics. When a new bar comes, the last calculation result is registered on the last tick and this result is shifted to the first bar in the history and does not change further. The system works in tickwise mode at the zero bar, but it does not mean that it moves back and forth like a weather vane, it works pro-actively and displays only significant movements of quotes trends. Although I would not call this system predictive, and I see from your questions that you are not accurately interpreting the examples I gave. When I was showing the forecast for 1 and 2 bars of the smoothed signal I was talking about one of the signals, that after filtering got smoothed but was falling behind by 2 bars, I wasn't talking about the forecast for another bar. Our objective was to compensate for the lag without affecting the signal smoothness. And all the models in the system are set to do that - to filter and immediately compensate for delays due to forecasts. When I was talking about the forecast horizon of one or two bars, I meant the horizon in relation to the initial signal whose phase has to be recovered, not in relation to the quotes flow. That is why you do not see in the charts where the forecast lines are shifted to the left relatively to the quotes, which begs the question - was there a forecast and what horizon are we talking about? By problem definition and implementation, this system wasn't trained as a predictor of quote flow (this problem is of completely different order, though also solvable, but much more complicated), the system is designed for MTS designing, and they don't need to know in advance what will happen, they don't need to mentally tune themselves to the upcoming events - have a cup of coffee or go to the bathroom, MTS should clearly, without delays, work out the current momentum. Hence, the system algorithm is as follows: create a presentable sampling of a group of signals (to increase the accuracy of the decision and eliminate false positives due to compensation for individual errors by a group decision) which, with a minimum delay determined by the time the system works (about several seconds), takes trading decisions of order management. So at every moment the system shows what is going on at the zero bar, at bar opening it shows how the trend will develop during the bar opening and at the last tick it shows where the trend will go further (of course, all this with a certain error and the system is far from being perfect), the signals synthesized by the system reflect slow trends and we may not always judge from them the development of a certain bar, but as I said, if the trend break occurs within a bar being formed, the system will not wait for the new bar to arrive, but as I said, the system will not wait for the new bar to be formed.
The other day I decided to make one more test of the system stability. I ran it on the daily data in the interval from August 1, 1992 till now. Since August 1, 1992 to January 26, 2000. - worked normally and stably in all market phases, since January 26, 2000, when the exchange rate went lower than 1.0037, the distortions started. On January 6, 2003, when the rate rose above 1.05, the system fully recovered its normal operation.
I have done one more experiment, I have tested the system on GBPUSD days from January 24, 1992 up to now, exchange rate fluctuations in this interval were from 1.3946 to 2.1161 - the system functions normally in the whole interval, though it was not trained on GBPUSD.
As a result, the system started working normally for EURUSD in the whole range (even with distortions), but in case of GBPUSD the system demonstrated small shift by a constant value in some market phases of the studied range, which had no significant effect on either signal form or phase. These experiments lead to the conclusion about the good stability of the system in a wide range of changes of the market quotes far beyond the learning curve, and the possibility of entering the system in the working range and restoring its performance by introducing the constant shift in quotes, in case of significant changes of the rate and appearance of distortions in operation. The system does not require any retraining or optimisation for its normal operation, as many TSs do, which require it to remain operational when the market phase changes.
While this system is far from being completed, there is still a lot of work to be done, and I have more and more doubts every day - will I complete it, or will it suffer the same fate as many others - die before it is even born. As a developer I suffer from one serious ailment: as you work on a project you build up experience, an understanding of how something can be made better, an endless series of improvements begins, often the point of no return is passed, and to make changes to what has been done is no longer possible, it's easier to throw everything away and start over again. As with this job, for the past three weeks it has been getting harder and harder to resist the temptation to throw the system in the bin and start afresh. Although, of course, it's a pity for the time and effort to just throw it away, even as it is now, it shows good results. But selling it in this way would be similar to selling a money-printing machine for the price of metal, the system will not be valued and will be considered as a simple indicator, although I had no other choice but to choose two complementary signals and attach the Expert Advisor for 200 quid, and the cost increases by orders of magnitude, although setup and testing will require a lot of time, which I don't have and which I do not want to do now because of impatience to start a new system.
So, I am in limbo, and the future of this system is murky. I hope I've answered the question as to why I haven't emptied the market yet?
In other words, the closer the cloud is to a 45-degree incline and the thinner it is, the better.
That's understandable, what I'm interested in now is the validity of the result.
result. Theoretically the angle could be more than 45, it's when it's always projected in the right direction, but that's overly optimistic.
Let's say we have a mixture of overwhelmingly random
and a small number of true but very optimistic predictions, then
the tangent would be pretty decent, and the variance of the "optimistic"
component won't have much effect on the variance. How do you derive such a result?
clear?
I'm thinking maybe I should try accidentally deleting 50% of the points
and look at the angle again, then remove it with another random set.
and so on a hundred times. If the resulting tangent is not very high and stays, on average, the same, it will be black and white.
and the average remains the same, then it's a fair prognosis. Otherwise it's a random
good luck. What do you think?Theoretically, the angle could be more than 45, which is when it is always projected in the right direction, but that's overly optimistic.
Your fears are unfounded - theoretically, this cannot be the case!
The point is that the range of possible values the slope tangent of the straight line can take in our case is limited to zero on one side and one on the other, and does not coincide with the range of values for the regular tangent. Judge for yourself, the function is smooth and defines "know nothing", this corresponds to zero and "know everything" to one. There is no other. One cannot know more than "everything"... Said, of course, refers to the limiting case where we have an "infinite" number of experiments. In reality, we are limited to a finite number of experimental data and, as a consequence, an inevitable statistical error may appear which in some cases may lead to values of tangent angles greater than 1. But there is nothing scary or unusual about this. After all, when we obtain an estimate for the angle, we will of course obtain an estimate of the error with which this value is found, and the "correct" result in your case will look like this
tan(a)= 0.9 +-0.3.
The result tan=1.1 does not therefore indicate an "optimistic forecast", it indicates a wrong representation of the result. You should simply specify the limits of the possible range of the value obtained.
Your fears are unfounded - theoretically, this cannot happen!
The point is that the range of possible values that the tangent of the slope of the straight line can take in our case is limited to "zero" on one side and "one" on the other, and does not coincide with the range of values for the regular tangent. Judge for yourself, the function is smooth and defines "know nothing", this corresponds to zero and "know everything" to one. There is no other. One cannot know more than "everything"... Said, of course, refers to the limiting case where we have an "infinite" number of experiments. In reality, we are limited to a finite number of experimental data and, as a consequence, an inevitable statistical error may appear which in some cases may lead to values of tangent angles greater than 1. But there is nothing scary or unusual about this. After all, when we obtain an estimate for the angle, we will of course obtain an estimate of the error with which this value is found, and the "correct" result in your case will look like this
tan(a)= 0.9 +-0.3.
So the result tan = 1.1 is not an "optimistic forecast", it is indicative of a wrong representation of the result, you just need to clarify the boundaries of the possible range of the value obtained.
It is not very clear how to "simply clarify the boundaries of the possible scatter of the value obtained". There are formal rules for constructing a straight line and estimating its slope,
the average degree of confidence in the forecast.
Suppose we have a number series {21;-5;12;23;-24}.
with it, then tan(a)=1, if our super-prediction system guesses the direction exactly, but the values
while predicting 2 times as much {42;-10;24;46,-48} then technically tan(a)=2, with a prediction of
3 times more than reality, the angle will run away beyond 70 degrees. If such forecasts
are a small percentage of the total number of forecasts and the rest are random, i.e. for
tan(a)=0, then the average alpha will be e.g. 10 degrees. How to deal with this value,
is it actually obtained by summing random and incorrect predictions?
Suppose we have introduced a rule that if the forecast exceeds the obtained value,
then consider it to be true and mark y with the same value as x. But then what to do if
it was forecasted 150 p upwards but in reality it went 5 p upwards?
Further, the fact that the angle from the bottom is limited to zero is also not true. If there is a super-anti-prediction system, i.e. it predicts an absolutely accurate future value, but opposite in sign, the angle would be -45. It is clear that in this case it is not difficult to reverse the direction. But what about if such predictions are also 'smeared' among a large number of random ones?
While this system is far from complete, there is still a lot of work to be done, and every day I have more doubts whether I will complete it, or whether it will suffer the fate of many others - to die before it is even born. As a developer I suffer from one serious ailment: as I work on a project experience grows, understanding of how something can be improved begins to appear, an endless series of improvements, often the point of no return is passed, and to make changes to the past is no longer possible, it is easier to throw it all out and start over again. As with this job, for the past three weeks it has been getting harder and harder to resist the temptation to throw the system in the bin and start afresh. Although, of course, it's a pity for the time and effort to just throw it away, even as it is now, it shows good results. But selling it in this way would be similar to selling a money-printing machine for the price of metal, the system will not be valued and will be considered as a simple indicator, although I had no other choice but to choose two complementary signals and connect the Expert Advisor for 200 quid, and the cost increases by orders of magnitude, although setup and testing will require a lot of time, which I don't have and which I do not want to do now because of impatience to start a new system.
So, I am in limbo, and the future of this system is murky. I hope I've answered the question of why I haven't emptied the market yet?
You make me laugh......I think it's time to get practical, otherwise theory won't do any good.......))))
about the slope angle, try to make a series of closures and a series of the same number of values, but from previous closures and you get an angle of about 45 degrees. Is that a prediction?
No. This is nonsense.
Are you forecasting an absolute value of price or price increments? If the absolute value, then everything is correct - almost an accurate prediction (tan=1). But the trouble is that what matters for trading is the "movement" of the price after opening a position, not its value. In other words, for example, for the "opened-closed" case on one bar, plot a series of Close-Open differences and a series of the same number of values, but from the previous Close-Open and you will get an angle of approximately zero degrees.
This is the prediction.
It's not very clear how to "just specify the limits of the possible variation of the value obtained". There are formal rules for drawing a straight line and estimating from its slope angle,
the average degree of confidence in the prediction.
Known as. If there is an equation of the least squares line drawn through the prognosis cloud, you can find the error of calculation of the slope tangent of this line. I can't give you the formula at a glance - I don't remember. I'll have to look it up in books.
...if our super-forecast system accurately guesses the direction
it predicts 2 times more {42;-10;24;46,-48} then technically tan(a)=2, with a prediction
3 times the reality, the angle will run away beyond 70 degrees.
Such a variant, except for making it up, cannot be realized in any other way!
The point is that during the construction of the prediction system we, during its adaptation (training), should be guided by a certain functional, which we will be able to minimize in one way or another. For example, every time you solve in your head the problem of how to cross a street, you minimize the possible risk associated with the busy traffic on it. Or, in another situation, the length of the distance travelled. These are possible functionalities. What is relevant for us (though not always) is minimizing the distance from a forecast point to its exact value (minimizing the root-mean-square error). In this case, your example is absurd, because the prediction is 2 (two) times different from reality, and it does not matter, that upwards. The important thing is that it is 2 times!
No need to make up a monster and then rack your brains about the possible world in which it could exist.
P.S.
Found a formula for determining the error of the tangent of the angle of inclination.
Suppose we are looking for an approximation by a linear function Y=a+bx, then b=(<xy>-<x><y>)/(<x^2>-<x>^2) and a=<y>-b<x> It is, in fact, how to find linear regression coefficients a and b if the set of predictions of the expected increment x[i] and the set of actual increments y[i] are known. The triangular brackets denote the mean, which is determined by the formula:
<x>=1/n*SUM(x[i]), where i spans values from 1 to n, n is the number of experimental points.
Then the error is defined as:
delta=SQRT((SUM((y[i]-b*x[i]-a)^2))/(n-2)*SUM(x[i]^2-n<x>^2))
The expression for the tangent of the angle of inclination, taking into account the above, takes the form:
tan=b+/-delta
No. This is nonsense.
Are you forecasting an absolute value of price or price increments? If the absolute value, then that's correct - almost an accurate prediction (tan=1). But the trouble is that it's the "movement" of the price after opening a position, not its value, that matters for trading. In other words, for example, for the "opened-closed" case on one bar, plot a series of Close-Open differences and a series of the same number of values, but from the previous Close-Open, and you will get an angle of approximately zero degrees.
This is the prediction.
It is known as. If you have the equation of the least-squares line through the forecast cloud, you can find the error in computing the tangent of the slope of the line. I can't give you the formula at a glance - I don't remember. I will have to look it up in books.And this is a thought.