Discussion of article "Practical application of neural networks in trading. Python (Part I)"

New comment
 

New article Practical application of neural networks in trading. Python (Part I) has been published:

In this article, we will analyze the step-by-step implementation of a trading system based on the programming of deep neural networks in Python. This will be performed using the TensorFlow machine learning library developed by Google. We will also use the Keras library for describing neural networks.

Let us consider some points related to data preparation for neural network training.

  • For decision making, we will use two neural networks for opening positions in one direction
  • According to the previous point, the training data should be divided into two groups - one for each direction.
  • As in the previous system, the first neural network will be trained to build indicators similar to standard technical indicators. We used this solution in the previous system because we used self-written indicators and we didn't want to overload the working Expert Advisor. Python is used because only quotes can be received from the terminal. To prepare data for the neural network, we need to build these indicators in a Python script. By teaching the neural network to build such indicators, we eliminate the need to duplicate them in the script. 
  • The second neural network builds the signal indicator, based on which we create a trading strategy.
  • The neural network will be trained for the EURUSD H1 chart.
  • As a result, to build the system, we will need to prepare two neural networks for buying and two networks for selling. Thus, four neural networks will work in the system.

Author: Andrey Dibrov

 

a quest, not an article

1.

there is no description of the constant 10 000 (1 and 4 zeros) in the text of the article or in the sources , the search on the article is used 51 times

I suspected that this is the number of digits for EURUSD, but then why 100 000 (1 and 5 zeros):

inB[15]=(iOpen(NULL,PERIOD_D1,iBarShift(NULL,PERIOD_D1,iTime(NULL,PERIOD_H1,i)))-iLow(NULL,PERIOD_D1,iBarShift(NULL,PERIOD_D1,iTime(NULL,PERIOD_H1,i))))*100000;
inB[16]=(iHigh(NULL,PERIOD_D1,iBarShift(NULL,PERIOD_D1,iTime(NULL,PERIOD_H1,i)))-iOpen(NULL,PERIOD_D1,iBarShift(NULL,PERIOD_D1,iTime(NULL,PERIOD_H1,i))))*100000;
inB[17]=(iHigh(NULL,PERIOD_D1,iBarShift(NULL,PERIOD_D1,iTime(NULL,PERIOD_H1,i)))-iLow(NULL,PERIOD_D1,iBarShift(NULL,PERIOD_D1,iTime(NULL,PERIOD_H1,i))))*10000;
inB[18]=(iHigh(NULL,PERIOD_D1,iBarShift(NULL,PERIOD_D1,iTime(NULL,PERIOD_H1,i)))-iOpen(NULL,PERIOD_H1,i+1))*10000;
inB[19]=(iOpen(NULL,PERIOD_H1,i+1)-iLow(NULL,PERIOD_D1,iBarShift(NULL,PERIOD_D1,iTime(NULL,PERIOD_H1,i))))*10000;


where do the numbers 10000 and 100000 come from? what do they mean?


2. PythonTestExpert

this code is a mystery in general:

void OnTick()
  {
   MqlDateTime stm;
   TimeToStruct(TimeCurrent(),stm);
   
   for(int i=0; i<=14; i++)
     {
      in[i]=in[i+5];
     }


I suspect that we shift the array every tick, but where do we get the initial values of the array from? .... below we fill this array from the 15th element to the end of the array.


3.

why is the code of indicators DibMin1-1.mq5 and DibMax1-1.mq5 identical by 99% separated in 2 indicators, and not executed in one indicator with 2 lines (buffers)?



4. on Python also questions, at first there was not a single comment to the Python script, nor the purpose of writing the script, but at the end of the article suddenly appeared a description of what the script was doing, and the fact that we have already used the data of this script when optimising the Expert Advisor.... well come back reader and look for where the data of this script was, maybe it was this:

While working on the article I decided to also give an example of another approach to NS learning objectives.

?




I wouldn't write that I didn't like the article ( or the article is absurd ?), but the title of the article "Practical Application of Neural Networks in Trading. Python" is very loud, and it's a pity that search engines will now display this article in the first search results

 
Igor Makanu:

It's a quest, not an article.

1.

there is no description of the constant 10 000 (1 and 4 zeros) in the text of the article or in the sources , the search on the article is used 51 times

It was suspected that this is the number of digits for EURUSD, but then why 100 000 (1 and 5 zeros):


where do the numbers 10000 and 100000 come from? what do they mean?


2. PythonTestExpert

this code is a mystery in general:


I suspect that we shift the array every tick, but where do we get the initial values of the array from? .... below we fill this array from the 15th element to the end of the array.


3.

why is the code of indicators DibMin1-1.mq5 and DibMax1-1.mq5 identical by 99%, but not executed in one indicator with 2 lines (buffers)?



4. on Python also questions, at first there was not a single comment to the Python script, nor the purpose of writing the script, but at the end of the article suddenly appeared a description of what the script was doing, and the fact that we have already used the data of this script when optimising the Expert Advisor.... well, come back reader and look where the data of this script was, maybe it was this:

While working on the article I decided to also give an example of another approach to NS learning objectives.

?




I wouldn't write that I didn't like the article ( or the article is absurd ?), but the title of the article "Practical Application of Neural Networks in Trading. Python" is very loud, and it's a pity that search engines will now display this article in the first search results

Thanks for the good questions!!!

1. 10,000 or 100,000 ? " * " or " / " ? etc. We form the input data for the NS. (The neural network that is in my skull visually liked the data that was output to the file. Maybe the neural network in the other box will like it too. Just kidding:) With NS, you have to experiment, both with the hyperparameters of the neural network itself, the architecture - and the input data. Here it's probably not "two times two is four" but "two times two is?"

2. On the question of PythonTestExpert. I missed the point that needed to be clarified - we run the Expert Advisor in the strategy tester on H1 at open prices. And the shift of the indicator cluster is done on each bar. In another case, I use the shift of indicators at 00 hours - once a day. Here, too, it is necessary to experiment.

3. Two indicators - purely for convenience and this is a special case. Often strategies are built on trading in any one direction. And, therefore, for data sampling it is more convenient to use one indicator line, which shows either the maximum extremum of the day, or the minimum. Here, too, experiment with the shift - "back and forth".

4. On the Python script - yes, maybe I wanted to simplify it too much..... But my goal in this article was to show the simplicity of practical application of neural networks using Python integration. At this stage - getting input data, training and testing. " Press the button and get the result". I don't know if this topic has been covered in this way? Not by me. By others. But I wanted to share it....

5. And as a joke on the issue of absurdity - isn't it absurd to look in a box at a set of formulas that have been called technical indicators and try to make money!!!)))).

 
Andrey Dibrov:

5. And as a joke on the question of absurdity - isn't it absurd to look in a box at a set of formulas that have been called technical indicators and try to make money!!!)))).

the article is absurd because it is impossible to sit down and read the material, there is no consistent presentation, no description of the methodology, no purpose of the study - I am not talking from simple to complex, there is simply no logic in following the material presented.

here it is

While working on the article, I decided to also give an example of a different approach to NS learning objectives. To do this, let's make some changes to the PythonIndicators script.

This is a sign that the article material is prepared in an inferior way, you want to explore additional possibilities of the given methodology - describe new goals, describe what we will use from the ready code

and in such a presentation it looks like, why don't we try to spin something else by the method of "scientific poking" - "I have been spinning and I know exactly what we will get:".

The logic here is that we are training the NS on goals that have been achieved, not on events that are just about to happen. Which, you may agree, is more logical. After all, it is easier to evaluate past events than to make forecasts.

As a result, we get such an indicator.


pro

Here are some results of optimisation.

I don't want to comment at all - a set of cutouts from Paint? Why are there different dates and sizes on the screens? Where are the values of optimisation parameters?

What is this? - Is it Google "search by picture"? It looks like this:


 
Igor Makanu:

the article is absurd because it is impossible to sit down and read the material, there is no consistent presentation, no description of the methodology, no purpose of the research - I am not talking from simple to complex, there is simply no logic in following the material presented.

this

While working on the article, I decided to also give an example of a different approach to NS learning objectives. To do this, let's make some changes to the PythonIndicators script.

This is a sign that the article material has been prepared in an inferior way, if you want to explore additional possibilities of the given methodology - describe new goals, describe what we will use from the ready code.

and in such a presentation it looks like, why don't we try to spin something else by the method of "scientific pumping" - "I have been spinning and I know exactly what we will get:".


about

I don't want to comment at all - a set of cut-outs from Paint ? Why are there different dates, different sizes on the screens ? Where are the values of the optimisation parameters ?

What is this? - Is that google "image search"? It looks like this to you:


In principle I agree with everything - I will finalise the article taking into account the comments. Thank you...

 
Andrey Dibrov:

In principle I agree with everything - I will finalise the article taking into account the comments. Thank you...

great!

the material is in demand, at least I am very interested in this direction.

ZY: yes here is more to my previous message - all research results in the article should have the effect of reproducibility, not only that it confirms the validity of the article, and you will not have questions where you got it (your screenshots of optimisation), well, and most likely it will be useful as a training material.


Sv.

[Deleted]  
have installed the MT5 package, but have never used it. It is possible to remove all indicators and trade from the python programme, for the sake of brevity
 
Maxim Dmitrievsky:
have installed the MT5 package, but have never used it. You can remove all indicators and trade from the python programme for brevity

Yes, we used the indicators to train NS in the python programme. And another python programme will then trade using the trained NS...

 
<br/ translate="no">

4. On the Python script - yes maybe I wanted to oversimplify..... But my goal in this article was to show the simplicity of practical application of neural networks using Python integration. At this stage - getting input data, training and testing. " Click on the button and get the result". I don't know if this topic has been covered in this way? Not by me. By others. But I wanted to share it....

5. And as a joke on the issue of absurdity - isn't it absurd to look in a box at a set of formulas called technical indicators and try to make money!!!)))).

"to show the simplicity of practical application of neural networks using Python integration" you failed. The procedure you propose cannot be called simple. And there is no integration at all .

"receiving input data, training and testing" - all this can and should be done in Python. The MetaTrader5 package was written for this purpose.

Without optimisation of neural network hyperparameters and quality preparation of input data, we cannot talk about any serious result. Python has everything to perform these important and not simple steps. Your article does not have it.

"Click a button and get a result" is not about your approach. In machine learning, there is a whole field developing - machine learning automation. There, yes, you collect the data, pass it to the program and it will determine the methods and techniques of preprocessing, select the model or models that give the best results for this data, optimise the hyperparameters and produce a ready result for use.

The article shows the wrong sequence of steps to create and use a neural network - this is a major flaw in my opinion. The fact that integration is not used in full is sad but not fatal. And you should be more modest with the title of the article.

Good luck

 
Vladimir Perervenko:

"toshow the simplicity of practical application of neural networks using Python integration" you failed. The procedure you propose cannot be called simple. And there is no integration at all .

"All this can and should be done in Python. The MetaTrader5 package was written for this purpose.

Without optimisation of neural network hyperparameters and quality preparation of input data, we cannot talk about any serious result. Python has everything to perform these important and not simple steps. Your article does not have it.

"Click a button and get a result" is not about your approach. In machine learning, there is a whole field developing - machine learning automation. There, yes, you collect the data, pass it to the program and it will determine the methods and techniques of preprocessing, select the model or models that give the best results for this data, optimise the hyperparameters and give the ready result for use.

The article shows the wrong sequence of steps to create and use a neural network - this is a major flaw in my opinion. The fact that integration is not used in full is sad but not fatal. And you should be more modest with the title of the article.

Good luck

If you have watched the video at the end of the article, you will notice that the whole process of data acquisition, NS training (on those hyperparameters) and construction of the resulting indicator takes up to 2 minutes. It is not difficult and available for any trader, not only for the "chosen ones".

On the issue of integration... The whole creative process takes place at the stages described in this article, especially on receiving input data and sampling of this data. Personally, I use my own indicators for this purpose. And now I will stress you a little)))) - at this stage of neural networks development, hyperparameters for analysing a pure time series are of little importance. Basically, NS are now "sharpened" for robotics (in a broad sense). For classification. That is, they work perfectly well on complete images. But if we train the NS on cats and dogs, and show it only an ear, then, exaggeratedly, we can get the answer that it is a pioneer in a cap. And, in our business, there are complete images only in the past, and in the present there are none.... That's why this stage of preparation for NS training is so important. And integration. - that's a purely technical process. Python simplifies it considerably and brings it to the plane of practical application in trading.

In machine learning, a whole direction is developing - automation of machine learning. There, yes, you collect the data, pass it to the program and it will determine the methods and methods of preprocessing, select the model or models that allow you to get the best results for this data, optimise the hyperparameters and produce a ready result for use.
Again this is suitable for classifying complete images. Where quality can be determined during training.... We have to keep coming back to testing, optimisation in MT5 etc. And here it is much more difficult to integrate, but as a perspective for future work, yes.....
The article shows wrong sequence of stages of creating and using neural network - this is the main drawback in my opinion. The fact that integration is not used to the full extent is sad, but not fatal. And you should be more modest with the title of the article.
What is the correct sequence?
 

After running two Scripts - PythonIndicators .mq5 , PythonPrices.mq5 empty .csv files are created - what could be the reason?

Thanks

12
New comment