Machine learning in trading: theory, models, practice and algo-trading - page 145

New comment
 
Alexey Burnakov:

It's a big problem. It's something that's almost impossible to solve without impairing learning.
The point here is that the more you look into the past, the more dependent are your neighboring observations.

If you feed the machine Not Sparse Observations, you lose the most important thing: meeting the requirement for independence of the observations. After that, learning will be drastically inadequate. And the statistics don't work.

And to avoid this you have to thin the observations so that the lag of maximal looking back is less or equal to the step of thinning observations. And this would lead to what? Hundredfold reduction of sample size for training.

We need a compromise here. Depth of look vs. number of examples in the sample.

I agree that's a problem, I've been graying over it for weeks...

What do you mean the more you look past the more dependent your neighboring observations are. ? doesn't make sense...

I also don't understand the essence of the problem...

 
mytarmailS:

I agree that it's a problem, I've been graying over it for weeks now...

what does it mean that the more you look into the past, the more addicted you are to neighboring observations. ? is not clear...

And please explain the slashing... I don't get it either...

Ok. Using your own example.

For two neighboring bars in time, you will draw almost 100% identical histograms of past prices. That is, the two training examples will be very similar just because they are close in time. This contradicts the basic property of statistical processes, where observations are independent. The machine will not learn chronologically-invariant patterns, but will learn repetitive ones (almost the same ones). Imagine you polled Vasya 200 times in an opinion poll and trying to extend the knowledge you found to 1,000 different people.
 
Alexey Burnakov:
Ok. On your example.

For two adjacent bars in time, you will draw almost 100% identical histograms of past prices. That is, the two training examples will be very similar just because they are adjacent in time. This contradicts the basic property of statistical processes, where observations are independent. The machine will not learn chronologically-invariant patterns, but will learn repetitive ones (almost the same ones). Imagine you polled Vasya 200 times in an opinion poll and trying to extend the knowledge you found to 1,000 different people.

Ooh!!! that's different, now everything is crystal clear :)

1) You can reduce the size of the window several times, then the sections will be less similar in each slice

2) You can maximally increasebreaks, which will also remove a little bit ofsmoothing

PRICE <- cumsum(rnorm(300))+1000
layout(1:2)
plot(PRICE,t="l",ylab = "цена" , xlab = "индексы",lwd=2)
Max <- max(PRICE)
Min <- min(PRICE)
range.vector <- seq(Min, Max, length.out=200)
H <- hist(PRICE, breaks = range.vector)
          

 
mytarmailS:

Ooh!!! that's different, now it all makes perfect sense :)

1) you can reduce the size of the window several times, then the sections will be less similar in each slicing

2) you can maximize thebreaks, that will also remove a littlesmoothing

I do it this way. Pseudocode:

Example1 = p[0]
step = 720
random = [0;50]
Example 2 = p[step+random]

where max lag to past = 720

That is I slice it so that observations are guaranteed not to overlap.

The same principle is needed to avoid dependence of the target from looking into the future.
 

The first thing to determine is what horizon you need. This will determine the depth of history you need to study.

Look at the trend charts with a history depth of 3000, 1000, 300 and 150 bars (AUDUSD/H1).

> library("PolyTrend", lib.loc="K:/RRO/R-3.3.1/library")
> trend3000 <- PolyTrend(price[ ,4], alpha = 0.05)
> require(magrittr)
Loading required package: magrittr
> trend1000 <- PolyTrend(price[ ,4] %>% tail(., 1000), alpha = 0.05)
> trend300 <- PolyTrend(price[ ,4] %>% tail(., 300), alpha = 0.05)
> trend150 <- PolyTrend(price[ ,4] %>% tail(., 150), alpha = 0.05)
> plot(trend3000, t="l")
> plot(trend1000, t="l")
> plot(trend300, t="l")
> plot(trend150, t="l")


I have inserted the pictures in reverse order.

And for 3000 bars the direction is "Up".

Trend type: 3 (cubic)
Slope: 0.0000
Direction: 1 (positive)
Significance: 1 (statistically significant)
Polynomial degree: 3 (cubic)

For the shorter ones

#  trend1000
PolyTrend classification:

Trend type: 2 (quadratic)
Slope: -0.0000
Direction: -1 (negative)
Significance: 1 (statistically significant)
Polynomial degree: 2 (quadratic)
#  trend300
PolyTrend classification:

Trend type: 2 (quadratic)
Slope: 0.0000
Direction: 1 (positive)
Significance: 1 (statistically significant)
Polynomial degree: 2 (quadratic)
#trend150
 PolyTrend classification:

Trend type: 3 (cubic)
Slope: -0.0000
Direction: -1 (negative)
Significance: 1 (statistically significant)
Polynomial degree: 3 (cubic)

If I'm interested in the forecast horizon of 24 bars (one day) I don't need to look backwards for 3000 bars. It is enough to look at 300 bars.

Good luck






 

I find the idea of predicting 24 steps ahead questionable.

In both classification and regression we should always predict exactly ONE step ahead, but for example on H1, H4 and D1. When trading on H1, we will get 4 and 24 steps ahead predictions for H1.

Ideally, the TF should be made artificially from the underlying quotient. In my example, we load H1 and do not take the rest from the terminal. Using this approach we will receive not D1 linked to 00:00, but strictly 24 hours ahead of the current hour

 
SanSanych Fomenko:

I find the idea of predicting 24 steps ahead questionable.

In both classification and regression we should always predict exactly ONE step ahead, but for example on H1, H4 and D1. When trading on H1, we will get 4 and 24 steps ahead predictions for H1.

Ideally, the TF should be made artificially from the underlying quotient. In my example, we load H1 and do not take the rest from the terminal. Using this approach we will not obtain D1 linked to 00:00, but strictly 24 hours ahead of the current hour

It depends on what we are forecasting. For example, a 24-hour trend forecast is good enough (the curve is very smooth). That is, when you need to see the FORM of the curve without any special requirements for accuracy.

Otherwise your example is absolutely correct (as a principle). And the sliding window option (24 hours) is also very workable. A lot depends on the details.

Good luck

 

Quick question...

I was making a distribution based on price, that's a usual vector, and I got a kind of market profile, and is it possible to build a real volume profile, that is to connect the price with the volume in the distribution?

 
mytarmailS:

Quick question...

I've been doing a price distribution, that's a usual vector, and I got a kind of market profile, but is it possible to build a real volume profile and connect the price with the volume in the distribution?

There are indicators that take the price and volume and build a general chart. And the distribution can be calculated for it. For example, Chaikin Oscillator or some more indicators in the Indicator/Volumes folder. But in forex will be tick volumes and not real ones. Although they correlate, it's still bad, it's better to look at the real volumes on the stock exchange.

 
Dr.Trader:

There are indicators that take the price and volume and build some general graph. And you can read the distribution for it. For example, Chaikin Oscillator or some more indicators in the Indicator/Volumes folder. It's better to look at real market volumes.

I don't trade forex, I'm not even familiar with Metatrader :)

I would like to experiment with the profile in its different types in the R-ka, I just do not understand how to build a distribution of two vectors

It's not necessary the volumes, it's just a start, you can put anything into the profile, but it has to be connected with the price and that means two vectors

1...138139140141142143144145146147148149150151152...3699
New comment