How to Start with MT5, a summary ! - page 8

New comment
 
Sergey Golubev #: Compiling MQL5 programs with AVX / AVX2 + FMA3 / AVX512 + FMA3 instruction set with build 3902

How will this affect Market products in the future?

I am assuming that the validation process will require that the product should be compiled without it to cater for the lowest common denominator.

 
Fernando Carreiro # : How will this affect Market products in the future? I am assuming that the validation process will require that the product should be compiled without it to cater for the lowest common denominator.

Forum on trading, automated trading systems and testing trading strategies

Compiling MQL5 programs with the AVX / AVX2 + FMA3 / AVX512 + FMA3 instruction set with build 3902

Renat Fatkhullin , 2023.09.01 13:24

Only programs compiled in "x64 Regular" mode will be accepted into the market.

Most likely in the future we will expand the opportunities in the market.

MT5 release is scheduled for Friday next week. Therefore, we ask you to actively test new modes.

 

Forum on trading, automated trading systems and testing trading strategies

Compiling MQL5 programs with the AVX / AVX2 + FMA3 / AVX512 + FMA3 instruction set with build 3902

Renat Fatkhullin , 2023.09.01 18:28

In the current beta 3905, available on MetaQuotes-Demo, we have expanded the information for MQL5:

 void OnStart ()
  {
   Print ( "CPU name:         " , TerminalInfoString (TERMINAL_CPU_NAME));
   Print ( "CPU cores:        " , TerminalInfoInteger ( TERMINAL_CPU_CORES ));
   Print ( "CPU architecture: " , TerminalInfoString (TERMINAL_CPU_ARCHITECTURE));   // новое
   Print ( "" );
   Print ( "EX5 architecture: " ,__CPU_ARCHITECTURE__);                            // новое
  }

CPU name:         12th Gen Intel Core i9-12900K
CPU cores:        24
CPU architecture: AVX2 + FMA3

EX5 architecture: AVX

You can find out on the fly the capabilities of the processor on which the terminal is running.

Using the string macro __CPU_ARCHITECTURE__ you can find out and check for which set of commands the EX5 file was compiled.


If the download fails, it says: 

your CPU architecture does not allow to run the file 'test.ex5', AVX512 required, you have AVX2

-------------------------

Something about MQL5 Cloud Network and MQL5 VPS in relation to AVX512 and AVX2 - 

Forum on trading, automated trading systems and testing trading strategies

Compiling MQL5 programs with the AVX / AVX2 + FMA3 / AVX512 + FMA3 instruction set with build 3902

Fernando Carreiro , 2023.09.01 19:15

What about the MetaQuotes VPS service? Will they all serve code compiled with extra instruction sets, or just regular compilation?

English original: What about MetaQuotes VPS service? Will they all also cater for code compiled with the extra instruction sets or only the regular compilation?

and the replies:

Forum on trading, automated trading systems and testing trading strategies

Compiling MQL5 programs with AVX / AVX2 + FMA3 / AVX512 + FMA3 instructions from build 3902

Renat Fatkhullin , 2023.09.01 19:22

In MetaQuotes VPS, we have modern processors with AVX2 everywhere. Therefore, migration will not be a problem.

But AVX512 versions will not be allowed in MQL5 Cloud Network or MetaQuotes VPS.

Forum on trading, automated trading systems and testing trading strategies

Compiling MQL5 programs with the AVX / AVX2 + FMA3 / AVX512 + FMA3 instruction set with build 3902

Renat Fatkhullin , 2023.09.06 08:41

By default everything is compiled in normal X64 mode. For those who know nothing and compile downloaded programs as is, nothing will change.

For those who want to move forward, there are all possibilities. Over the course of two releases, all terminals will automatically be divided into three versions: x64, avx and avx2 - this is written in the start banner of the terminal. For a couple of years now, terminals have been distributed in two versions, x64 and avx.

When distributing programs, including the market, you need to build the x64 version. The remaining options are for your controlled use. Finding out what set of instructions is in a vps server is not a problem.

At MetaTrader VPS Hosting, the last few servers without AVX2 support will be replaced with modern ones by the end of the week and it will be possible to migrate x64/avx/avx2 programs to our cloud without any problems. On third-party VPS, just check the brand of the processor, use the CPU-Z program or look at the start banner of the terminal.

An explicit transition to specialized versions of terminals for modern processors is a big and correct step. We are seriously focused on improving the performance of the entire platform and now we have a free hand to directly implement new optimization methods without building crutches.

Forum on trading, automated trading systems and testing trading strategies

Compiling MQL5 programs with the AVX / AVX2 + FMA3 / AVX512 + FMA3 instruction set with build 3902

Renat Fatkhullin , 2023.09.09 16:58

MetaTrader VPS Hosting is completely on AVX2 versions.

MetaTrader 5 terminals in the cloud are launched immediately in AVX2 builds: 

Windows Server 2022 build 20348 on Xen, 32 x AMD EPYC 7542 32 -Core, AVX2 , 61  / 71 Gb memory, 149  / 255 Gb disk, admin, GMT+ 1

-------------------------


Forum on trading, automated trading systems and testing trading strategies

Compiling MQL5 programs with the AVX / AVX2 + FMA3 / AVX512 + FMA3 instruction set with build 3902

Renat Fatkhullin , 2023.09.06 13:07

In MQL5 Cloud Network, the upper limit for EX5 will be AVX2 and tasks will be automatically distributed to agents with the required processor set.

That is, everything will work transparently in a couple of days. Tomorrow we are releasing an open beta for brokers, and a week later there will be a release.


Forum on trading, automated trading systems and testing trading strategies

Compiling MQL5 programs with the AVX / AVX2 + FMA3 / AVX512 + FMA3 instruction set with build 3902

Renat Fatkhullin , 2023.09.06 13:58

...

We are planning a major upgrade of the MQL5 compiler with more complete support for the AVX2/512 instruction set over the next two months. Plus optimization of the internal functions of the language, which will give good acceleration.

 

visual studio

The question:

Forum on trading, automated trading systems and testing trading strategies

Compiling MQL5 programs with AVX / AVX2 + FMA3 / AVX512 + FMA3 instructions from build 3902

Renat Akhtyamov , 2023.09.07 12:40

Good afternoon, the developers!

what does install visual studio mean and what will it allow, if not write in C# for example?

..and the reply:

Forum on trading, automated trading systems and testing trading strategies

Compiling MQL5 programs with the AVX / AVX2 + FMA3 / AVX512 + FMA3 instruction set with build 3902

Renat Fatkhullin , 2023.09.07 13:07

Once you install Visual Studio, you can compile C/C++ files directly into MetaEditor.

For example, this allows you to easily compile EXE files or DLL libraries directly within MQL5 projects/directories:


------------------------

The announcement:

MetaTrader 5 Platform Beta Build 3930: Deposits/withdrawals in the terminal and support for AVX instructions to speed up programs


MetaTrader 5 Platform Beta Build 3930: Deposits/withdrawals in the terminal and support for AVX instructions to speed up programs
MetaTrader 5 Platform Beta Build 3930: Deposits/withdrawals in the terminal and support for AVX instructions to speed up programs
  • 2023.09.07
  • www.mql5.com
The beta version of the updated MetaTrader 5 platform is to be released on Friday, September 8, 2023...
 

Forum on trading, automated trading systems and testing trading strategies

New MetaTrader 5 Platform build 3980: Improvements and fixes

MetaQuotes, 2023.09.21 15:01

The updated version of the MetaTrader 5 platform will be released on Thursday, September 21, 2023. It features a number of platform fixes and improvements.

New MetaTrader 5 Platform build 3980: Improvements and fixes


MetaTrader 5 Client Terminal build 3980

  1. Terminal: New trading report improvements. Fixed the display of the total swaps value and the profit chart by symbols.
  2. Terminal: Optimized deposit and withdrawal pages. For further details about the new platform integration with payment systems, please read the build 3950 release notes.
  3. Terminal: Optimized recalculations of financial operations across the entire platform, including the strategy tester. Now profit, margins, and many other parameters are calculated faster.
  4. Terminal: Updated user interface translations.
  5. MQL5: Added Conjugate methods for complex, vector<complex> and matrix<complex> types. They implement complex conjugate operations.
    //+------------------------------------------------------------------+
//| Script program start function                                    |
//+------------------------------------------------------------------+
void OnStart()
  {
   complex a=1+1i;
   complex b=a.Conjugate();
   Print(a, "  ", b);
   /*
   (1,1)  (1,-1)
   */

   vectorc va= {0.1+0.1i, 0.2+0.2i, 0.3+0.3i};
   vectorc vb=va.Conjugate();
   Print(va, "  ", vb);
   /*
   [(0.1,0.1),(0.2,0.2),(0.3,0.3)]  [(0.1,-0.1),(0.2,-0.2),(0.3,-0.3)]
   */

   matrixc ma(2, 3);
   ma.Row(va, 0);
   ma.Row(vb, 1);
   matrixc mb=ma.Conjugate();
   Print(ma);
   Print(mb);
   /*
   [[(0.1,0.1),(0.2,0.2),(0.3,0.3)]
    [(0.1,-0.1),(0.2,-0.2),(0.3,-0.3)]]

   [[(0.1,-0.1),(0.2,-0.2),(0.3,-0.3)]
    [(0.1,0.1),(0.2,0.2),(0.3,0.3)]]
   */
   
   ma=mb.Transpose().Conjugate();
   Print(ma);
   /*
   [[(0.1,0.1),(0.1,-0.1)]
    [(0.2,0.2),(0.2,-0.2)]
    [(0.3,0.3),(0.3,-0.3)]]
   */
  }
  6. MQL5: Added handing of ONNX model outputs of the 'Sequence of maps' type.

    For ONNX models that provide Map sequences in the output layer (ONNX_TYPE_SEQUENCE of ONNX_TYPE_MAP), a dynamic or fixed array of structures should be passed as the output parameter. The first two fields of this structure must match the ONNX_TYPE_MAP key and value types and be fixed or dynamic arrays.

    Consider the iris.onnx model created by the following Python script:
    from sys import argv
data_path=argv[0]
last_index=data_path.rfind("\\")+1
data_path=data_path[0:last_index]

from sklearn.datasets import load_iris
iris_dataset = load_iris()
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(iris_dataset['data'], iris_dataset['target'], random_state=0)
from sklearn.neighbors import KNeighborsClassifier
knn = KNeighborsClassifier(n_neighbors=1)
knn.fit(X_train, y_train)

#  Convert into ONNX format
from skl2onnx import convert_sklearn
from skl2onnx.common.data_types import FloatTensorType
initial_type = [('float_input', FloatTensorType([None, 4]))]
onx = convert_sklearn(knn, initial_types=initial_type)
path = data_path+"iris.onnx"
with open(path, "wb") as f:
    f.write(onx.SerializeToString())
    Open the created onnx file in MetaEditor:


    Viewing the ONNX model in MetaEditor


    The Map sequence is passed as "output_probability". It has a key of INT64 type (which corresponds to long in MQL5) and the float type value. To receive data from this output, declare the following structure: 
    struct MyMap
  {
   long              key[];
   float             value[];
  };
    Here we used dynamic arrays with appropriate types. In this case, we can use fixed arrays because the Map for this model always contains 3 key+value pairs.

    Since the Map sequence is returned, an array of such structures should be passed as a parameter for receiving data from output_probability output. This array can be dynamic or fixed, in accordance with the properties of a particular model. Example:
    //--- declare an array to receive data from the output layer output_probability
MyMap output_probability[];

...

//--- model running
OnnxRun(model,ONNX_DEBUG_LOGS,float_input,output_label,output_probability);
  7. MetaEditor: Fixed display of output types in the ONNX model viewer.

MetaTrader 5 Web Terminal build 3980

  1. Added Contact Broker section in the web terminal's main menu.
  2. Added error handling for SSL authentications. This authentication type is not supported in the web terminal. One-time passwords can be used instead.
  3. Fixed desktop platform download link in the main menu.
  4. Fixed accounts managing dialog. If the broker does not provide the demo or real account opening option, the relevant menu item will be hidden.


The update will be available through the Live Update system.


 

ONNX and  MQL5 Copilot


The forum

  1. MetaEditor, Open AI and ChatGPT - summary thread
  2. Learning ONNX for trading - key forum thread about ONNX
  3. Practical examples: this page

    CodeBase

    1. Information about the ONNX model's inputs and outputs - script for MetaTrader 5
    2. Get info about inputs and outputs of onnx-model - script for MetaTrader 5

    Documentation

    1. ONNX models in machine learning (MT5)

    Learning ONNX for trading - the video:

    1. ONNX Runtime - post  #60
    2. Converting Models to #ONNX Format - post  #61
    3. ONNX – open format for machine learning models​ - post #62
    4. (Deep) Machine Learned Model Deployment with ONNX - post  #63
    5. Recurrent Neural Networks | LSTM Price Movement Predictions For Trading Algorithms- post 
    6. Artificial Intelligence Full Course | Artificial Intelligence Tutorial for Beginners | Edureka - post  #230
    7. Open Neural Network Exchange (ONNX):
      7.1. Introduction to ONNX - Tutorial 1 - post
      7.2. Challenges in Deep Learning - Tutorial 2 - post
      7.3. All about ONNX - Tutorial 3 - post
      7.4. Design principles - Tutorial 4 - post
      7.5. ONNX file format - Tutorial 5 - post
      7.6. ONNX Data Type - Tutorial 6 - post
      7.7. Machine Learning Example - Tutorial 7 - post
      7.8. ONNX Runtime - Tutorial 8 - post
      7.9. ONNX Model Zoo - Tutorial 9 - post
      7.10. ONNX Model Zoo Demo - Tutorial 10 - post
      7.11. PyTorch to Tensorflow Demo - Tutorial 11 - post
    8. MIT Introduction to Deep Learning:
      8.1. Introduction to Deep Learning - Lecture 1 - post
      8.2. Recurrent Neural Networks and Transformers -  Lecture 2 - post
      8.3. Convolutional Neural Networks - Lecture 3 - post 
      8.4. Deep Generative Modeling -  Lecture 4 - post
      8.5. Reinforcement Learning -  Lecture 5 - post
      8.6. Deep Learning New Frontiers -  Lecture 6 - post
      8.7. LiDAR for Autonomous Driving -  Lecture 7 - post
      8.8. Automatic Speech Recognition -  Lecture 8 - post
      8.9. AI for Science -  Lecture 9 - post
      8.10. Uncertainty in Deep Learning -  Lecture 10 - post

    =======================

    Evaluating ONNX models using regression metrics 

    Evaluating ONNX models using regression metrics

    Regression is a task of predicting a real value from an unlabeled example. A well-known example of regression is estimating the value of a diamond based on such characteristics as size, weight, color, clarity, etc.

    The so-called regression metrics are used to assess the accuracy of regression model predictions. Despite similar algorithms, regression metrics are semantically different from similar loss functions.

    Evaluating ONNX models using regression metrics
    Evaluating ONNX models using regression metrics
    • www.mql5.com
    Regression is a task of predicting a real value from an unlabeled example. The so-called regression metrics are used to assess the accuracy of regression model predictions.
     
    Thanks for this summary!
     

    Forum on trading, automated trading systems and testing trading strategies

    How to start with MQL5

    Sergey Golubev, 2024.03.23 06:33

    Master MQL5 from beginner to pro (Part I): Getting started with programming

    Master MQL5 from beginner to pro (Part I): Getting started with programming

    I sometimes receive private messages from those who want to learn how to create their own Expert Advisors or indicators. Although there is a lot of material on this site and on the Internet in general, including very good resources with examples, beginners still need help. Some users seek more consistency in presentation, others require clarity or something else. Sometimes users ask: "Add comments to the code of a working Expert Advisor, I will understand everything and make the same one myself!" But, after they see a hundred lines of code, they get scared, confused and give up the idea. Instead, they say: "Better create an Expert Advisor for me."

    Anyway, there are people who still want to "understand everything". This is when a step-by-step guide, explaining things from simple to complex, may come in handy. Well, users can't find such a step-by-step guide for the MQL5 language.

    Master MQL5 from beginner to pro (Part I): Getting started with programming

    I am starting this series of articles specifically for beginners, for those who are encountering programming for the first time and are willing to learn it. These articles are intended for traders who no longer want to depend on programmers and wish to understand how everything works. The articles offer consistent and detailed information, with pictures and examples, guiding readers to the level of "MQL5 language proficiency".


    12345678
    New comment
    Reason: