Not Mashka's business! - page 6

 
to Neutron

Oh dear, our models are slightly different. A little question - in order to smooth out the differences, can you go from predicting the "ideal MA" to predicting the quotes, i.e. using your MA to "restore" the quotes?


And let me remind you of a few more things:

  • my forecast horizon is selected adaptively and the model doesn't allow entering it, otherwise it would lose its meaning
  • the value of a moving window of my MA is also selected adaptively in each interval
 

You can't do without the great Al!

You have to think, and how to do it I'm done... going to bed.

 

If you don't "break" the models, I see the following

  • predict quotes
  • predict classic MA with lag (I confess I'm getting confused with terminology: ideal MA, dip depth, ... :o)).

Or I'll have to dig through everything and predict the ideal MA, but it's not soon and I don't really need it. The model assumes to generate such an MA only for the current countdown, kind of like "adaptive filtering".


PS: ok, we'll figure it out tomorrow, but I'll wait with ale for now... :o)

 

OK, let's predict the quote!

Any method and algorithm can be used. The only requirement is that the code must not "look into the future". This requirement is not as trivial to implement as it may seem at first sight. For instance, it may be of a hidden nature related to the processing of High and Low. These values change during the bar construction up to the moment when it is closed, therefore processing High and Low on the history is tantamount to "peeping". I suggest to use only opening prices and build a forecast cloud not by the absolute values of BP, but by the increments of the forecast and initial series.

Besides the forecast of a price series itself is a VERY difficult task, as a consequence we will be comparing the value of slope angle of the regression line in area of zero (up to 10%). So I suggest to build a regression for Venera BP first and make sure we get zero with better than 1% accuracy on a sample of 10000 predictions, that will be a criterion of code correctness. Yes, predict correctly 1 bar ahead and as a quotient I propose to take the Open Minute series (as the most predictable) of EURGBP pair the year 2004 for example.


What do you think?


Here is what the prediction problem looks like using the common (lagged) MA with a 10-bar window:

On the left side you see row of minutae and its converging MA with prediction one bar ahead, in the centre - abscissa axis - row of Open prices: dBid[i]=(Open[i]-Open[i-1]), ordinate axis - increments of the MA. Blue line is a linear regression drawn on the forecast cloud. Its slope shows the quality of the forecast, if its tangent is equal to 1 or 100% (45 degrees) we have a completely accurate forecast, if it tends to zero we have no forecast. Just above the graph above, the tangent of the slope is 3%, i.e. it does not predict! The thing is I used a statistic of 1000 forecasts, the error will be 1/SQRT(1000), i.e. just around 3%.

The figure on the right shows prognostic ability (tangent) of the lagged MA as a function of window width in bars. You can see that the predictive value of the tool tends to zero as the averaging window increases!

 

Generally agree, but...

Можно использовать любой метод и алгоритм его реализации. Неприменное требование - код не должен "заглядыват" в будущее. Это требование не так тривиально выполнить, как может показаться на первый взгляд. Например, это может носить скрытый характер связанный с обработкой High и Low. Эти величины изменяются в процессе построения бара вплоть до момента его закрытия, поэтому обработка на истории High и Low равносильна "подглядыванию". Предлагаю использовать только цены открытия и строить прогнозное облако не по абсолютным значениям ВР, а по значениям приращений прогнозного и исходного рядов.

To retrieve data, I use theGetHistoryProcess(cb, window) function, which excludes peeking far into the future. The GetFutureProcess(cb, window) function is used to check the result.


Function parameters:

  • cb - current count
  • window - sample size

Here is their code:


Data source

GetHistoryProcess(cb, window)

GetFutureProcess(cb, window)


It is assumed that the "current bar" is the bar at the time it is fully formed. In other words, a quote is 'about to come in' which will become open.

Besides, the forecast of the price series itself is a VERY difficult task, as a consequence we will be comparing the value of the slope of the regression line in the area of zero (up to 10%). So I suggest to build a regression for Venera BP first and make sure we get zero with better than 1% accuracy on a sample of 10000 predictions, that will be a criterion of code correctness. Yes, predict correctly 1 bar ahead and as a quotient I propose to take a series of Open Minutes (as the most predictable) of EURGBP pair the year 2004 for example.

You can also use Wiener process, but what difference does it make - you will still have the same problem, the current bar has not "fully" formed.

Here is what the prediction problem looks like using the common (lagged) MA with a 10-bar window:

You have to decide what to do with the horizon, I have it adaptively selected. Maybe we should add an additional characteristic, for example - total number of forecast samples or average forecast length, something like that.

On the left side we have the line of minutes and its declining MA predicting one bar ahead, in the centre - the abscissa axis is the series of increments of Open prices: dBid[i]=(Open[i]-Open[i-1]), the ordinate axis is the increments of the MA. Blue line is a linear regression drawn on the forecast cloud. Its slope shows the quality of the forecast, if its tangent is equal to 1 or 100% (45 degrees) we have a completely accurate forecast, if it tends to zero we have no forecast. Just above the graph above, the tangent of the slope is 3%, i.e. it is not predicting! The thing is, I used a statistic of 1000 predictions, the error will be at 1/SQRT(1000) i.e. just in the region of 3%.

I disagree, let's plot the scatter plot by prices only (by absolute values). I don't like the increment. You have to understand that if we start from increments, they will be concentrated near zero. Try to do a scatterplot of prices for your random process.


Addendum: My opinion is that the absolute value of price or MA should be predicted. Do you agree?

 
grasn:

It is assumed that the 'current bar' is the bar at the time it is fully formed. In other words, a quote is "about to come in" and will become open.

You can do it on the Wiener process, but who cares - you'll still have the same problem, the current bar hasn't "properly" formed.

So, to be clear, we build BPs ONLY from Open.


We need to decide what to do with the horizon, I have it adaptively selectable. Maybe we should add some extra feature, like total number of forecast samples or average forecast length or something like that.

This is not part of the task. You may work with any horizon you want, but don't look ahead and don't use anything but bar opening prices. But! You should only use "1 bar forward" as a forecast vector.


I disagree, let's build a scatter plot using only prices (absolute values). I don't like the increment. You have to understand that if you're relying on increments, they'll be concentrated near zero. Try to do a scatterplot of prices for your random process.

I'm sorry, I don't understand you here.

Our task is to predict a series of opening prices. So do it! Then, having obtained this series with 10,000 samples, we will compare it with the original one. And to estimate effectiveness of the forecast, let's construct two series of increments and immediately see the forecast failures. Simple. Clearly!

Am I wrong? What's the use of comparing washed-out integral curves. Look again at my pictures in the post above. The first picture shows a smooth curve that curves around a series of quotes... And just analyzing the number of their increments (second fig.) it becomes clear that it is of NO use. Sergey, it is a good criterion. Don't fight it.

We grown-ups don't need rose-coloured glasses :-)

 

Это не входит в задачу. Работай с каким хочешь горизонтом, только не заглядывай прирасчётах в будущее и не используй ничего кроме цен открытия бара. Но! прогнозный вектор построй только из расчёта "1 бар вперёд".


Here's where I don't understand you (in bold what exactly), clarify what this is, i.e. we're giving up on the "full forecast vector".


I will not look into the future, you have my word of honour.

Our job is to predict a series of opening prices. So do it! Then when we get this series with 10,000 samples, we compare it with the initial one. And to estimate forecast efficiency let's construct two series of increments and see forecast failures immediately. Simple. Clearly!

Am I wrong? What's the use of comparing washed-out integral curves. Look again at my pictures in the post above. The first picture shows a smooth curve that curves around a series of quotes... And just analyzing the number of their increments (second fig.) it becomes clear that it is of NO use. Sergey, it is a good criterion. Don't fight it.

Try to plot and then compare the scatter plots of increments and absolute MA values. Theoretically, the slope angle and the amount of scatter relative tolinear regression lines should be the same for each plot. And look at the result as real guys look through a colt sight :o) Will it be the same for you?


Important

Let's be more specific. So, we've already decided - we're forecasting CLOSE, all that's left to do is figure it out:

  • predict the CLOSE series itself
  • or some MA(my window size is adaptive for each prediction, it seems to be the same for you) which are built on this CLOSE.

If the price itself (i.e. CLOSE), it will take me time to correct the "price reconstruction" function by MA. If we analyze the MA forecast, I think I will launch a small test-drive today, for 10-50 bars (about one minute is considered for one count)

 

To avoid arbitrariness in choosing Open, Close,(H-L)/2 I use analysis of tick-flow, + the value of the prediction I consider not in the number of bars, but in time. And I think the "ideal MA" is not the one you're using. You should use Fourier transform (PF), then thresholding in the frequency domain and inverse PF, the result is an ideal curve with the most plausible estimate in the middle of the window. And then what you are trying to do.

 

to grasn

Close is Close. It's a deal.

We really did not understand each other about the prediction vector. Well, let's see.

I won't repeat about "scatter diagram of increments and absolute values", everything is clear here.


to Prival

There are certain difficulties with ticks - you need a big history, preferably without holes, etc. These requirements are easier to implement in archives with hours or minutes.

As for the perfect curve, so let's compare the two-run MEMA (that's what I use) and what the Fourier-smoothing will give. I suggest that the criterion for "goodness" is the value of the standard deviation from the cotier and the smoothness of the curve itself - the smaller the cotier and smoother the curve - the steeper!

 

to Neutron

Close так Close. Договорились.

It would be easier for me to MA on CLOSE values. Alright, while I'm building the "price restorer for MA" I'll do a couple of tests for MA. By the way, you didn't show the data for prices anywhere, you only showed the prediction of increments for MA. Do you have this "restorer"?

Regarding the "scatter plot of increments and absolute values" I won't repeat myself, it's obvious.

The inverted commas could have been removed, it is the official name in statistics of what we build. In vain you ignore my question, but that's your business. If you'd built it, you'd know the significant difference. It is exactly the area of non-obvious things.


to Prival


And it seems we have already roughly defined the parameters:

  • The slope of the linear regression
  • Standard deviation - we will use it to estimate "the value of dispersion" relatively to the straight line. We can also determine the slope
Reason: