Machine learning in trading: theory, models, practice and algo-trading - page 197

 
ivanivan_11:
just wondering - do you only take price values and different indicators from prices as predictors here? and does anyone use real volumes and indicators from volumes?

I work with prices, without volume.

I tried to use volumes and their indicators, but it didn't work. The real volumes in forex are not given, but there are ticks, which seem to correspond a little bit to the real ones, that is, in principle, they can be tried. But the problem is that the history of bars and ticks from the broker contains some "not such" tick volumes. While the symbol is in the Market Watch in the running terminal - a correct history of tick volumes is obtained for it. If the symbol is removed from the chart or the terminal is closed - the tick volumes for such bars will be taken according to those provided by the broker. And these two values, "typed by the terminal itself" and "received from the broker" are quite different, sometimes ten times. Now I need to keep the terminal running for a couple of months to get the real tick volumes, not what broker gives me, then I can try to use them again.

 
Dr.Trader:

I work with prices, without volume.

I tried to use volumes and their indicators, but it didn't work. The real volumes in forex are not given, but there are ticks, which seem to correspond a little bit to the real ones, that is, in principle, they can be tried. But the problem is that the history of bars and ticks from the broker contains some "not such" tick volumes. While the symbol is in the Market Watch in the running terminal - a correct history of tick volumes is obtained for it. If the symbol is removed from the chart or the terminal is closed - the tick volumes for such bars will be taken according to those provided by the broker. And these two values, "typed by the terminal itself" and "received from the broker" are quite different, sometimes ten times. Now I need to keep the terminal running for a couple of months to get real tick volumes and not broker's ones and then I can try to use them once again.

I'm not talking about exotics like Brazil, South Africa, Poland, the United Arab Emirates, but I'm sure they have real volumes of futures and stocks in Bourgesundia.
 
Renat Fatkhullin:

For information on the R language and the new MetaTrader 5 build 1467:

  • An updated version of statistical libraries similar to R has been released:

    Statistical Distributions in MQL5 - take the best from R and make it faster

  • Calculations in MQL5 are from 3 to 7 times faster than in R (even taking into account that functions there are implemented in C++).
  • Some R functions have errors due to old optimization/simplification methods, leading to erroneous results
  • Added a beta-version of graphical libraries similar to R, which gives the possibility to visualize the data as in R.
  • Added useful function ArrayPrint, which prints both standard arrays and structures similar to R


You can update to 1467 from the MetaQuotes-Demo server.

In coming versions we will add a lot of new mathematical and statistical functions, similar to R. It will allow you to do more calculations and visualizations directly in MetaTrader 5.

Dear Renat!

Let me respond to the comment on point 6. The detected errors of calculations in R at the following link:Statistical Distributions in MQL5 - taking the best from R and making it faster

The check has been performed for the Gamma function. We will leave the checking of other functions to your own responsibility.

On behalf of Data Science department, represented by the head and two statisticians (I can inform you in the PM), we inform you that the density of gamma distribution at zero point is not reduced to zero, strictly speaking.

R does estimate the density at the point zero to be 1:

x <- seq(0, 20, 0.5) #support

plot(k, dgamma(x = x, shape = 1, scale = 1, log = FALSE)) #PDF plot

print(dgamma(x = 0.000001, shape = 1, scale = 1, log = FALSE)) #limiting to zero

However, we can see that when x tends toward zero, the density approaches unity.

Moreover, according to:

https://en.wikipedia.org/wiki/Gamma_distribution

when x = 0, alpha = 1, beta = 1, we get an indeterminate value in the numerator, which brings the whole fraction into uncertainty.

We declare that, strictly speaking, the gamma distribution density at point zero is undefined. And when the limit on the right is taken, the density is equal to one.

In light of this, we think that the wording of the statement "calculation errors in R" is not correct. More precisely, it is a matter of convention: what counts as equal to the expression zero to the power of zero. Equating the gamma distribution density to zero at the point zero does not seem to be any conditional practice.

Sv.

Alexey

Gamma distribution - Wikipedia
Gamma distribution - Wikipedia
  • en.wikipedia.org
Gamma Parameters Support PDF CDF Mean Median Mode Variance Skewness Excess kurtosis Entropy MGF CF With a shape parameter k and a scale parameter θ. With a shape parameter α = k and an inverse scale parameter β = 1/θ, called a rate parameter. With a shape parameter k and a mean parameter μ = k/β. In each of these three forms...
 
Alexey Burnakov:

We declare that, strictly speaking, the density of the gamma distribution at the point zero is undefined. And when the right-hand limit is taken, the density is equal to one.

In light of this, we believe that the formulation of the statement "calculation errors in R" is not correct. More precisely, it is a matter of convention: than to equate the expression zero to the power of zero. Equating the gamma distribution density to zero at the point zero does not seem to be any conditional practice.

If the density (pdf) is nonzero, then the integral (cdf) at that point must also be nonzero, otherwise zero is not possible.

> n <- 5> k <- seq(0,1,by=1/n)> gamma_pdf<-dgamma(k, 1,1, log = FALSE)> gamma_cdf<-pgamma(k, 1,1, log = FALSE)> k[1] 0.0 0.2 0.4 0.6 0.8 1.0> gamma_pdf[1] 1.0000000 0.8187308 0.6703200 0.5488116 0.4493290 0.3678794> gamma_cdf[1] 0.0000000 0.1812692 0.3296800 0.4511884 0.5506710 0.6321206

But cdf=0. This is hard to explain.

At x=0 the pdf is zero by definition:


Wolfram Alpha:

For non-central and central chi-squared distributions:


Wolfram|Alpha: Computational Knowledge Engine
  • www.wolframalpha.com
Wolfram|Alpha is more than a search engine. It gives you access to the world's facts and data and calculates answers across a range of topics, including science, nutrition, history, geography, engineering, mathematics, linguistics, sports, finance, music...
 
Quantum:

If the density (pdf) is nonzero, then the integral (cdf) at that point must also be nonzero, otherwise zero is not obtained.

But cdf=0. This is hard to explain.

At x=0 pdf is zero by definition:


Wolfram Alpha:

For non-central and central chi-squared distributions:


You'd better tell me from yourself what 0 ^ alpha - 1 equals, when alpha = 1.

The integral isn't defined there either.... But in the limit it is close to zero. Question of Conventions...

print(pgamma(q = 0.000001, shape = 1, scale = 1, log = FALSE, lower.tail = F)) #right side

print(pgamma(q = 0.000001, shape = 1, scale = 1, log = FALSE, lower.tail = T)) #left side

 
ivanivan_11:
and does anyone use real volumes and volume indicators?
See https://www.mql5.com/ru/forum/86386/page174#comme nt_2911488
Машинное обучение: теория и практика (торговля и не только)
Машинное обучение: теория и практика (торговля и не только)
  • www.mql5.com
Добрый день всем, Знаю, что есть на форуме энтузиасты machine learning и статистики...
 

For the limit to be defined, it needs to be the same for left and right approaches, but this is not the case:

http://www.wolframalpha.com/input/?i=Limit%5BPDF%5BGammaDistribution%5B1,1%5D,x%5D,+x-%3E0%5D


Wolfram|Alpha: Computational Knowledge Engine
  • www.wolframalpha.com
Wolfram|Alpha is more than a search engine. It gives you access to the world's facts and data and calculates answers across a range of topics, including science, nutrition, history, geography, engineering, mathematics, linguistics, sports, finance, music...
 
Quantum:

For the limit to be defined, it needs to be the same for left and right approaches, but this is not the case:

http://www.wolframalpha.com/input/?i=Limit%5BPDF%5BGammaDistribution%5B1,1%5D,x%5D,+x-%3E0%5D


There is only one on the right and that is 1...

What Wolfram says 0 is not the truth in the last resort. I mean, the word "error" in the wording is redundant...

 

I found another error in R. R does not divide by 0 correctly.

Here is the script:

//+------------------------------------------------------------------+
//|                                                         test.mq5 |
//|                        Copyright 2016, MetaQuotes Software Corp. |
//|                                             https://www.mql5.com |
//+------------------------------------------------------------------+
#property copyright "Copyright 2016, MetaQuotes Software Corp."
#property link      "https://www.mql5.com"
#property version   "1.00"

#property script_show_inputs

input double div = 0.0;

//+------------------------------------------------------------------+
//| Script program start function                                    |
//+------------------------------------------------------------------+
void OnStart()
  {
//---
   Print(1.0/div);
  }
//+------------------------------------------------------------------+

The correct answer, in mql is.
zero divide in 'test.mq5' (20,13)
with
script stop

Unprivileged, in R:
> 1/0
Inf
with
script continuation

I mean the same thing as Alexey - the behavior of programs under undefined conditions may vary, and this is not an error. The way the architecture is supposed to be, that's how the result will be.


 

Another interesting observation. You can't divide by zero. And you can't take the root out of negative numbers. I remember from uni that all this is possible, but that's not the point now, in simple math you can't do both actions.

Why if I divide by 0 in mql then it breaks the whole script and breaks it, but if I extract the root of a negative number then I get the number "-nan(ind)" without breaking the script, and this "-nan(ind)" can be used to make further calculations (it will break all the following calculations)? Here I expected the same behavior of mql script in both cases.

Reason: