Discussion of article "Deep Neural Networks (Part I). Preparing Data" - page 3

 
Vladimir Perervenko:

In ggplot2(v2.2.1) the definition of geom_candlestick (MRO 3.4.1) disappeared.

I have already torn down MRO 3.4.0 in which I did all the calculations so I will find a solution tomorrow and write.

What version of R do you have ?

Thanks!

The latest version is "R version 3.4.2 (2017-09-28)"

 

That's why I don't like this ggplot2. Here's a working variant that I didn't include in the article.

#--------quantmod----------------------------
require(quantmod)
require(timetk)
evalq(
  pr %>% tk_xts(.) %>%
        chartSeries(x = OHLC(.), 
                    #c("auto", "candlesticks", 
                    #"matchsticks", "bars","line")
                    type = "bars", 
                    subset = 'last 3 days',#weeks, months
                    show.grid = T,
                    name = "EURJPY M15",
                    tyme.scale = T,
                    log.scale = FALSE,
                    line.type = "l",
                    bar.type = "ohlc",
                    theme = chartTheme('white',
                                       up.col = 4, 
                                       dn.col = 2,
                                       grid.col = 3,
                                       main.col = 1,
                                       sub.col = 4), 
                    major.ticks = "day", 
                    minor.ticks = TRUE ,
                    plot = TRUE,
                    color.vol = F,
                    multi.col = F
                    ),
      env)

It looks like this.

timetable

Good luck

 
Vladimir Perervenko:

That's why I don't like this ggplot2. Here's a working variant that I didn't include in the article.

It looks like this.


Good luck.

Thank you!

Your variant builds a chart normally in RStudio, but in MT4 terminal it does not work. I'm struggling for the second day and the chartSeries command does not work through the env environment.

If possible, please share an Expert Advisor for MT4 that unloads quotes and builds a chart. Thank you.

I could only do it the old way, as I did before. It is very inconvenient to write all commands in the terminal, not in R.


Rv(R, "Data",tm);

Rv(R, "Time",tm);

Rv(R, "Open",o);

Rv(R, "High",hi);

Rv(R, "Low",lo);

Rv(R, "Close",clo);

Rv(R, "Volume",vol);


PREDICTION_COMMAND=

"library(magrittr) "+CR

+"library(dplyr) "+CR

+"library(xts) "+CR

+"library(quantmod) "+CR

+"price <- cbind(Time = rev(Time), Open = rev(Open), High = rev(High), Low = rev(Low), Close = rev(Close)) "+CR

+"price_t <- price "+CR

+"dts = price_t[,1]"+CR

+"mydates = structure(price_t[,1],class=c('POSIXt','POSIXct')) "+CR

+"price_time <- xts(x=price_t[,c(2:5)], order.by=mydates, tzone='GMT') "+CR

;

RExecuteAsync(R,PREDICTION_COMMAND);

Rx(R, "chartSeries(price_time, type = 'bars', subset = 'last 3 days',show.grid = T,name ='EURUSD M15',tyme.scale = T,log.scale = FALSE,line.type = 'l',bar.type = 'ohlc',theme = chartTheme('white',up.col = 4,dn.col = 2,grid.col = 3,main.col = 1,sub.col = 4),major.ticks = 'day',minor.ticks = TRUE,plot = TRUE,colour.vol = F,multi.col = F)");


 
Konstantin Kopylov:

Thank you!

Your variant builds a chart normally in RStudio, but in MT4 terminal it does not work. The second day I'm struggling and I can't get the chartSeries command through the env environment.

If possible, please share an Expert Advisor for MT4 that unloads quotes and builds a chart. Thank you.

I could only do it the old way, as I did before. It is very inconvenient to write all commands in the terminal, not in R.


Rv(R, "Data",tm);

Rv(R, "Time",tm);

Rv(R, "Open",o);

Rv(R, "High",hi);

Rv(R, "Low",lo);

Rv(R, "Close",clo);

Rv(R, "Volume",vol);


PREDICTION_COMMAND=

"library(magrittr) "+CR

+"library(dplyr) "+CR

+"library(xts) "+CR

+"library(quantmod) "+CR

+"price <- cbind(Time = rev(Time), Open = rev(Open), High = rev(High), Low = rev(Low), Close = rev(Close)) "+CR

+"price_t <- price "+CR

+"dts = price_t[,1]"+CR

+"mydates = structure(price_t[,1],class=c('POSIXt','POSIXct')) "+CR

+"price_time <- xts(x=price_t[,c(2:5)], order.by=mydates, tzone='GMT') "+CR

;

RExecuteAsync(R,PREDICTION_COMMAND);

Rx(R, "chartSeries(price_time, type = 'bars', subset = 'last 3 days',show.grid = T,name = 'EURUSD M15',tyme.scale = T,log.scale = FALSE,line.type = 'l',bar.type = 'ohlc',theme = chartTheme('white',up.col = 4,dn.col = 2,grid.col = 3,main.col = 1,sub.col = 4),major.ticks = 'day',minor.ticks = TRUE,plot = TRUE,colour.vol = F,multi.col = F))");


Afternoon.

I do not plot in this way from MT. It is cumbersome and not convenient. Interactive and any other charts should be built from R via shiny.

In the V part of the article I will attach an Expert Advisor with the output of a simple chart. But I don't understand why you need a chart of quotes?

I intended to output a chart of test results, real-time trading results and a breakdown by time, symbol, etc. Maybe I won't do everything in this Expert Advisor.

For working with time/date look at the more convenient timekt package. Your choice.

Good luck

 

Hello, Vladimir,

1) why

v.rstl = c(NA, diff(rstl)*10)

is multiplied by 10, but the others are not?

With normalisation the multiplication effect - will disappear.

2) Digital filters are counted by Close prices. At the beginning of the bar the Close price is unknown (and equal to Open). And counting by Close - you get peeking into the future, which slightly increases accuracy.
Maybe you should count filters by Close of the previous bar or at least by Open of the current bar?

For the experiment I tried to count filters by Open - the results became worse by a few per cent.
According to the experiment from the 6th article

> #---5-----best----------------------
[1] 0.677 0.674 0.673 0.672 0.669 0.669 0.668

instead of the Close [1] 0.720 0.718 0.718 0.718 0.715 0.713 0.713 0.712.

3) Why are the filters FATL, SATL, RFTL, RSTL excluded from further calculations? The calculation is only on oscillators. I tried to leave them in, but clusterSim considered them important and did not filter them out. As a result of the training I got an error of about 50%, i.e. although they are important, they significantly worsen the result.
I guess it makes sense to use only oscillators as inputs for neural networks?
 
elibrarius:

Hello, Vladimir,

1) why

is multiplied by 10, but the others are not?

With normalisation the multiplication effect - will disappear.

2) Digital filters are counted by Close prices. At the beginning of the bar the Close price is unknown (and equal to Open). And counting by Close - you get peeking into the future, which slightly increases accuracy.
Maybe you should count filters by Close of the previous bar or at least by Open of the current bar?

For the experiment I tried to count filters by Open - the results became worse by a few per cent.
According to the experiment from the 6th article

> #---5-----best----------------------
[1] 0.677 0.674 0.673 0.672 0.669 0.669 0.668

instead of the Close [1] 0.720 0.718 0.718 0.718 0.715 0.713 0.713 0.712.

3) Why are the filters FATL, SATL, RFTL, RSTL excluded from further calculations? The calculation is only on oscillators. I tried to leave them in, but clusterSim considered them important and did not filter them out. As a result of the training I got an error of about 50%, i.e. although they are important, they significantly worsen the result.
I guess it makes sense to use only oscillators as inputs for neural networks?

Good afternoon.

1. This predictor had very small values and could fall out during normalisation. I simply fitted it into the range of the other predictors. With the SpatialSign method, the predictors should not differ by orders of magnitude.

2. All quote values are taken from the formed bars starting from 1. In general, it is better to count a zigzag by High/Low. In the last articles I have provided variants of ZZ calculation.

3. These 4 are continuous lines and are not applicable as inputs. The first difference over them is applicable

v.fatl = c(NA, diff(fatl)),
v.rftl = c(NA, diff(rftl)),
v.satl = c(NA, diff(satl)),
v.rstl = c(NA, diff(rstl)*10))

Digital filters have one important advantage over other indicators - they are non-parametric (conventionally speaking of course). But I like them very much.

Good luck

 

Hello, as following your article (1st time touching R), right in the 2nd brick of code I was faced with the folllwing error :

Error in evalq({ : object 'env' not found

does env here means something named in the pc software ? Or it is really a object that was created automatically?


Awesome article, gonna find a way to surpass it, would be awesome if you could help :)

(Using RStudio and installed MRO 3.5.3 (because 3.4.0 was outdated))

 
ferox875 :

Hello, following your article (touching R for the first time), right in the second block of code, I ran into the following error:

Error in evalq ({: object 'env' not found

here env means something named in pc software? Or is it really an auto-generated object?


Great article, I will find a way to beat it, it would be great if you could help :)

(Using RStudio and installing MRO 3.5.3 (since 3.4.0 is deprecated))

The env object was created to separate data from various tools. Just at the beginning of the script write

env <- new .env()
ls(env)
character( 0 )
env$a <- 23
ls(env)
[ 1 ] "a"
> env$a
[ 1 ] 23 

Good luck.

 

Dear MR. Vladimir Perervenko, thank you a lot for your fast answer, after taking a R course I hope to make at least enough to deserve the study of your awesome work, thank you a lot for sharing Mr. Vladimir.



Best of best Regards


Ferox

 

Hello again Mr. Perervenko, hope you are feeling great. I have a new question, when you first wrote about ZigZag :

#------ZZ-----------------------------------
par <- c(25, 5)
ZZ <- function(x, par) {
# x - vector
  require(TTR)
  require(magrittr)
  ch = par[1] 
  mode = par[2]
  if (ch > 1) ch <- ch/(10 ^ (Dig - 1))
  switch(mode, xx <- x$Close,
         xx <- x$Med, xx <- x$Typ,
         xx <- x$Wd, xx <- x %>% select(High,Low))
  zz <- ZigZag(xx, change = ch, percent = F, 
               retrace = F, lastExtreme = T)
  n <- 1:length(zz)
  for (i in n) { if (is.na(zz[i])) zz[i] = zz[i - 1]}
  return(zz)
}

on the 9th line, what's the meaning of the Dig object?

Couldn't find it on the project or in the required packages ...


Best Regards MR. Perervenko


Ferox