Machine learning in trading: theory, models, practice and algo-trading - page 201

 
SanSanych Fomenko:

Dear colleague!

For several pages there is an argument about the differences between your and R's algorithms on the edges of the definition of functions. Extreme points are extreme points and in practice the differences could be neglected.

But in this case I have a much more substantial question:

Where is the documentation for all your functions?

Previously, I thought we would take your function, then take the documentation for R, since your functions are analogues, and delve into those parts of R's documentation that either describe the algorithms or go to the links provided by R. R has very high quality documentation and reference apparatus.

In the course of the argument, I found out that your functions are different from R - they are some other functions whose algorithms rely on other sources. There is nothing in the article itself about this, no documentation. And we learn about it from Renat in a completely different context.

In practice, it follows unambiguously that the porting of the code from R to MQL5 is impossible.

And here's why.

It's because you haven't read the article and have missed the repetition several times:

  1. We have attached unit-tests in the source code, proving the correctness of calculations.
  2. We have done a lot of work on the total check of correctness that allows you to get to the bottom of R's errors
  3. We constantly double-checked the results in MQL5, Wolfram Alpha and R in our work
Unlike you, we have parsed the source code of R, analyzed each function in details, accelerated some functions up to 46 times, wrote an article and got ready to answer for it.

But instead critics manage to make statements on the old yeast of their knowledge and confidence in R without thoughtful reading of the article itself. And certainly without trying to run test scripts with evidence.

 

So that there is no misunderstanding:

  1. The translation/similarity of the specified R functions is correct
  2. The use of the logic from R in MQL5 is practically identical, and the code in MQL5 is clearer
  3. All the code of the mentioned functions in MQL5 is presented in the clean source code, including the check unit tests (where are they in R?)
  4. The calculations in MQL5 are times faster than in R.
  5. Our programmers are good at math and are as good (or better) than those who write R.
This is the beginning of a long way towards implementing complex mathematics in the standard MQL5 library. Now we have Alglib, Fuzzy and Stat.

Today we will release a graphics library similar to the one in R. There is already a functional expansion of Stat library mat functionality in the form of dozens of new functions.

 
Quantum:

At the moment there is a description of the functions in the article https://www.mql5.com/ru/articles/2742


Thank you, I understand your point of view.

PS.

As I understand, you have seen the documentation on Normal and other functions in R, so I will not make a copypaste and compare with your link.

 
Renat Fatkhullin:

This is because you have not read the article and missed the repetition several times:

  1. We have attached unit-tests in the source code proving the correctness of the calculations
  2. We have done a lot of work on the total check of correctness, which allows you to get to the bottom of R errors
  3. We constantly double-checked the results in MQL5, Wolfram Alpha and R in our work
Unlike you, we have parsed the source code of R, analyzed each function in details, accelerated some functions up to 46 times, wrote an article and got ready to answer for it.

But instead critics manage to make statements on the old yeast of their knowledge and confidence in R without thoughtful reading of the article itself. And certainly without trying to run test scripts with evidence.

Thank you, your point of view is clear to me.

PS.

This is the second time you reproach me, that I did not read the article.

Yes, I did, moreover I found error in function (returned scalar instead of vector), which you have corrected and now this error gone.

 
SanSanych Fomenko:

Thank you, your point of view is clear to me.

PS.

This is the second time you reproach me for not reading the article.

Moreover, I found a mistake in the function (it returned a scalar instead of a vector), which you have corrected and now there is no such mistake.

This is a counter measure to your comment "In practice, the conclusion that the porting of code from R to MQL5 is impossible".

Perhaps, you have read the old article and not the new one from yesterday. There was no error in it - there was a version with return of scalar (a long time ago) and we added vector functions. The library has seriously grown since the old article and was essentially rewritten.

I was right - you didn't read the new article.

I hope there will be no more talks about "you don't have these functions and you made them up yourself".
 
Alexey Burnakov:

Regarding the t distribution. I reproduced this error on the latest version of R. But I did not get into the essence of the algorithm. I admit that the correction was really needed.

I will try to ask a question to the R support team.

Article "Computing discrete mixtures of continuous distributions" is available on the author's website.

The authors of the article say that the problem is in the convergence criterion of the series.


The implementation of their proposed recurrence algorithm 7.2 https://github.com/neurodebian/afni_removeme_eventually/blob/master/nct.c

However, the recurrence calculation has uncertainties. For example, for the beta function:


#include <Math\Stat\Math.mqh>
//+------------------------------------------------------------------+
//| Script program start function                                    |
//+------------------------------------------------------------------+
void OnStart()
  {
//---
   double a=1;
   double b=1;
   double r_beta=MathBeta(a,b);
   for(int j=0; j<40; j++)
     {
      if(j>0)
        {
         r_beta*=((a+j-1)/(a+b+j-1));
        }
      double beta=MathBeta(a+j,b);
      PrintFormat("%d   error=%5.20e",j,beta-r_beta);
     }
  }

2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 0 error=0.00000000000000000000e+00
2016.11.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 1 error=-5.55111512312578270212e-17
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 2 error=0.00000000000000000000000000e+00
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 3 error=1.11022302462515654042e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 4 error=1.38777878078144567553e-16
2016.11.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 5 error=-2.22044604925031308085e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 6 error=2.22044604925031308085e-16
2016.11.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 7 error=1.66533453693773481064e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 8 error=-2.91433543964103591861e-16
2016.11.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 9 error=-1.24900090270330110798e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 10 error=-2.77555756156289135106e-17
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 11 error=-1.24900090270330110798e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 12 error=5.68989300120392726967e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 13 error=-2.91433543964103591861e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 14 error=-5.55111512312578270212e-17
2016.11.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 15 error=-4.85722573273505986435e-17
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 16 error=-1.87350135405495166196e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 17 error=5.27355936696949356701e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 18 error=-1.04083408558608425665e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 19 error=-3.400058018291454190505e-16
2016.11.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 20 error=-3.26128013483639733749e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 21 error=4.57966997657877072925e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 22 error=-3.19189119579732505372e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 23 error=1.52655665885959024308e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 24 error=5.55111512312578270212e-17
2016.11.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 25 error=-6.24500451351650553988e-17
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 26 error=-2.70616862252381906728e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 27 error=4.85722573273505986435e-17
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 28 error=4.64905891561784301302e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 29 error=-8.32667268468867405318e-17
2016.11.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 30 error=-9.02056207507939689094e-17
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 31 error=-2.98372437868010820239e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 32 error=2.22044604925031308085e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 33 error=2.74086309204335520917e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 34 error=-3.43475248243407804694e-16
201611.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 35 error=2.08166817117216851329e-17
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 36 error=-4.16333634234433702659e-17
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 37 error=8.673617379889403547206e-17
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 38 error=-1.21430643318376496609e-16
2016.11.11 12:28:30.015 RecurrentBeta (EURUSD,H1) 39 error=-2.18575157973077693896e-16

For the calculation of CDF its accuracy may be enough, but for the circulation of quantiles with great accuracy it may not be enough.

Therefore, we used direct summation without recurrence calculations in the quantile algorithm.

 
Renat Fatkhullin:

This is a counter-measure to your comment "In practice, there is a clear conclusion that the transfer of code from R to MQL5 is impossible".

I hope there won't be any more talk about "you don't have these features and you made them up".

Before writing about something I have a bad habit to read. "I've never made inline comments; I've commented only on the essence of my subject.

So it turns out that I cannot make my point of view clear to you, and it's not the first time.

Therefore, without me.

I sincerely wish you luck, Renat!

 

Gentlemen, what is the argument for 100,500 pages? Who knows more about math? What difference does it make to you as a matter of principle if the function is defined in 0 or not, if the article says there is an error in R or not. It's as if they are discussing your child here and saying that he is full of shit, while you stand up for him with your whole chest.

Take it as a given and use built-in features in MT5, ask developers for new features, if something is missing, write your own. You can use R from mt5 through crutches, but luckily they exist - mt4r.

 
Thanks for your participation @SanSanych Fomenko, but you failed to refute any of our positions and the real outline failed.
 
Renat Fatkhullin:

So that there is no misunderstanding:

  1. The translation/similarity of the specified R functions is correct
  2. The use of the logic from R in MQL5 is practically identical, and the code in MQL5 is clearer
  3. All the code of the mentioned functions in MQL5 is presented in the clean source code, including the check unit tests (where are they in R?)
  4. The calculations in MQL5 are times faster than in R.
  5. Our programmers are good at math and are as good (or better) than those who write R.
This is the beginning of a long way towards implementing complex mathematics in the standard MQL5 library. Now we have Alglib, Fuzzy and Stat.

Today we will release a graphics library similar to the one in R. The Stat library mat functionality is already being extended with dozens of new functions.

it's just a holiday
Reason: