Machine learning in trading: theory, models, practice and algo-trading - page 2282

 
mytarmailS:

Do R Connect

already did, but it didn't work outhttps://www.mql5.com/ru/forum/306688/page10#comment_11308158

 
Renat Fatkhullin:
Can you share the information?
1) Do you use the python library of MT5?
2) Do you use it outside or inside MT5?
3) What functions does the library lack? Access to indicators?

We are preparing an upgrade of MQL5 adding fast matrix operations. This will allow to perform massive calculations normally.

Further we will develop connectors to analytical packages and implement the standard WinML integration.
1) make a normal integration with sqlite3
2) display the error when the script stopped abruptly, like in python, with a link to the line and a description of the error and that would be by default in the log
3) add an integration with api telegram, at least a minimal set
4) allow the placing on the market advisers with webrequest, ie with the ability to request information from the outside
 
Evgeny Dyuka:
1) make a normal integration with sqlite3
2) display the error when the script stopped abruptly, like in python, with a link to the line and an error description, and by default send it to the journal
3) add integration with api telegram, at least a minimal set
4) allow to put on the market advisors with webrequest, ie with the ability to request information from the outside.

1) it has long existed as a standard DatabaseXXX functions, even a sqlite database browser is built into the editor, a detailed article about SQL in MQL5

2) the entire output of python scripts is written in logs in detail

3) this has long existed as separate solutions and does not apply to python library. I asked questions about the Python link. Instead of telegram it's convenient to use the regular pushers in cell phones

4) has always been available

SQLite: нативная работа с базами данных на SQL в MQL5
SQLite: нативная работа с базами данных на SQL в MQL5
  • www.mql5.com
Разработка торговых стратегий связана с обработкой больших объемов данных. Теперь прямо в MQL5 вы можете работать с базами данных с помощью SQL-запросов на основе SQLite. Важным преимуществом данного движка является то, что вся база данных содержится в единственном файле, который находится на компьютере пользователя.
 
Igor Makanu:

it already happened, but it didn't work outhttps://www.mql5.com/ru/forum/306688/page10#comment_11308158

sadly....

the most popular product in the world after matlab for statistical calculations, call it a collective farm )))

The level is clear and so is the collective farm...

 
mytarmailS:

sadly....

the most popular product in the world after matlab for statistical calculations, call it a collective farm )))

The level is clear, and so is where the collective farm is...

The level is shown here: Statistical Distributions in MQL5 - Taking the Best of R and Making it Faster

Comparison of Calculation Speed

We have prepared scripts for measuring calculation time of density functions (pdf), probability functions (cdf), functions for computing quantiles and functions for generating pseudorandom numbers to compare the calculation speed of statistical functions.

The calculations were performed on an array of 51 values; for continuous distributions the function values were calculated within the range of 0 to 1, for discrete ones - from 0 to 50. The calculation time of the statistical functions in R has been measured using the microbenchmark library. The function calculation time in MQL5 was calculated using the GetMicrosecondCount() function. The TestStatBenchmark.mq5 calculation script can be found in the terminal_data_folder\MQL5\Scripts\UnitTests\Stat. The script for R and the results of calculation speed measurements are given in the Appendix.

The calculations were performed on Intel Core i7-4790, 3.6 Ghz CPU, 16 GB RAM, Windows 10 x64.

Calculation time measurement results (in microseconds):


Distribution
MQL5 time
calculation PDF (µs)
R calculation time
PDF (microseconds)
PDF
R/MQL5
MQL5 time
CDF calculation (µs)
R time
CDF calculation (µs)
CDF
R/MQL5
MQL5 calculation time
quantiles (µs)
R calculation time
quantile (µs)
Quantile
R/MQL5
MQL5 Generation time
random numbers (μs)
R generation time
random numbers (μs)
Random
R/MQL5
1
Binomial
4.39
11.663
2.657
13.65
25.316
1.855
50.18
66.845
1.332
318.73
1816.463
5.699
2
Beta
1.74
17.352
9.972
4.76
15.076
3.167
48.72
129.992
2.668
688.81
1723.45
2.502
3
Gamma
1.31
8.251
6.347
8.09
14.792
1.828
50.83
64.286
1.265
142.84
1281.707
8.973
4
Cauchy
0.45
1.423
3.162
1.33
15.078
11.34
1.37
2.845
2.077
224.19
588.517
2.625
5
Exponential
0.85
3.13
3.682
0.77
2.845
3.695
0.53
2.276
4.294
143.18
389.406
2.72
6
Uniform
0.42
2.561
6.098
0.45
1.423
3.162
0.18
2.846
15.81
40.3
247.467
6.141
7
Geometric
2.3
5.121
2.227
2.12
4.552
2.147
0.81
5.407
6.675
278
1078.045
3.879
8
Hypergeometric
1.85 11.095
5.997
0.9
8.819
9.799
0.75
9.957
13.28
302.55
880.356
2.91
9
Logistic
1.27
4.267
3.36
1.11
4.267
3.844
0.71
3.13
4.408
178.65
626.632
3.508
10
Weibull
2.99
5.69
1.903
2.74
4.268
1.558
2.64
6.828
2.586
536.37
1558.472
2.906
11
Poisson
2.91
5.974
2.053
6.26
8.534
1.363
3.43
13.085
3.815
153.59
303.219
1.974
12
F
3.86
10.241
2.653
9.94
22.472
2.261
65.47
135.396
2.068
1249.22
1801.955
1.442
13
Chi Square
2.47
5.974
2.419
7.71
13.37
1.734
44.11
61.725
1.399
210.24
1235.059
5.875
14
Noncentral ChiSquare
8.05
14.223
1.767
45.61
209.068
4.584
220.66
10342.96
46.873
744.45
1997.653
2.683
15
Noncentral F
19.1
28.446
1.489
14.67
46.935
3.199
212.21
2561.991
12.073
1848.9
2912.141
1.575
16
Noncentral Beta
16.3
26.739
1.64
10.48
43.237
4.126
153.66
2290.915
14.909
2686.82
2839.893
1.057
17
Negative Binomial
6.13
11.094
1.81
12.21
19.627
1.607
14.05
60.019
4.272
1130.39
1936.498
1.713
18
Normal
1.15
4.267
3.71
0.81
3.983
4.917
0.7
2.277
3.253
293.7
696.321
2.371
19
Lognormal
1.99
5.406
2.717
3.19
8.819
2.765
3.18
6.259
1.968
479.75
1269.761
2.647
20
T
2.32
11.663
5.027
8.01
19.059
2.379
50.23
58.596
1.167
951.58
1425.92
1.498
21
Noncentral T
38.47
86.757
2.255
27.75
39.823
1.435
1339.51
1930.524
1.441
1550.27
1699.84
1.096
<PDF R/MQL5>
3.474 <CDF R/MQL5>
3.465
<Quantile R/MQL5>
7.03
<Random R/MQL5>
3.13

Table 4. Time required to calculate statistical functions in R and MQL5 (in microseconds).

For R we took minimum time values, and for MQL5 we took average values (pdf_mean, cdf_mean, quantile_mean, random_mean).

The table shows that even under such conditions the calculation of functions of MQL5 statistical library is several times faster than R. On average, MQL5 calculates 3 to 7 times faster than R even taking into account that the compared versions of R functions are in fact written in C++.

In practice, the MQL5 language compiler has turned out to be much faster than C++ implementations of functions in R, which shows the high quality of our developments. When transferring programs from R to MQL5, you can get a significant speed-up and you do not need to use third-party DLLs.


MQL5 is now hundreds of times faster than Python, but with the introduction of native matrix operations as the basis for the language, we will see the speed of mathematics, that is several times faster than third-party Python modules.

 
Renat Fatkhullin:

1) has long been available as a regular DatabaseXXX functions, even the sqlite database browser is built into the editor, a detailed article about SQL in MQL5

2) the entire output of python scripts is written in logs in detail

3) this has long existed as separate solutions and does not apply to python library. I asked questions about the Python link. Instead of telegram it's convenient to use the regular pushers in cell phones

4) has always been available

Thanks for the detailed answer, I somehow missed the DB
 

Speed is good, but far from decisive

When you titled your article "Statistical Distributions in MQL5 - Using the Best of R and Making it Faster" I

I swear I laughed all day long! You took three functions from millions of ones in R, invented them and rewrote them into MQL5 and that's it! ))) All of you have surpassed R!!!! Ahahaha )))) can't excuse me...


What about the fact that R has the largest database of any statistical methods, for any direction, thousands of libraries... When will you rewrite it?

What about the full list of any MO algorithms, hundreds of libraries... When are you going to rewrite it?

What about the CRAN database, where every day the world's latest products appear, which everyone wants and can try... When are you going to rewrite it? And most importantly WHY?

Do you know that I can replace a hundred µl in R with one line of code?


All you have in your arguments is speed, the speed of the three functions you took from R.

Did you know you can write any C++ function in R?


Eh, okay... Why am I making a big deal about it... It's useless anyway, you've already solved everything... and so have I...

One thing I ask - don't write anymore that "you take something better from R", you don't know what you're talking about...


Even if you rewrite a thousand functions on µl it will not be even a hundred percent of what is in R, what's the point of doing this foolishness?

Why not let people themselves port into this world of statistics, machine learning, signal processing, econometrics and much more...

 
mytarmailS:

Speed is good, but far from decisive.

When you titled your article "Statistical Distributions in MQL5 - Taking the Best of R and Making It Faster", I laughed the whole day!

I swear I laughed all day long! You took three functions from the millions in R, invented them and rewrote them into MQL5 and that's it! ))) All of you have surpassed R!!!! Ahahaha )))) can't excuse me...

We took and rewrote in pure MQL5 about 500 built-in R functions. That's what is the basic mathematical basis of R system. All the rest are third-party packages.

We have rewritten these functions faster and distribute them in the /include/math/stat source code. This shows the quality of our language and our developers.

We have made our choice as to who and how to support them:

  • develop MQL5
  • Have integration with Python.


When you're throwing questions and rants, I recommend that you look at your opponent, who:

  • has been relentlessly writing code for 30 years
  • spent half of his life developing 5 trading platforms
  • Has created, maintains and fills this trading community for 15 years.
  • ...search for yourself at your leisure

If you're here to consume (no one here has seen your results in public), then behave decently and watch your language.
 
Renat Fatkhullin:
If you came to consume (no one saw your results here in public), then behave decently and watch your speech.

I don't consume anything here, I'm just communicating on the forum, if you ban me it will only make me feel better... So...

And these 500 functions that you rewrote, they're dead in the sense that they're like a ruler with which you can measure something, and this "something" is exactly the third-party packages to which you locked access

 
Renat Fatkhullin:

Would it make sense to make ZeroMQ native support?

Reason: