Practical application of neural networks in trading

Andrey Dibrov | 29 September, 2020

Introduction

This article considers the application of neural networks when creating trading robots. This is the narrow sense of this problem. More broadly, we will try to answer some questions and to address several problems:

1. Can a profitable system be created using machine learning?
2. What can a neural network give us?
3. The rationale for training neural networks for decision making.
4. Neural Network: is it difficult or simple?
5. How to integrate a neural network into a trading terminal?
6. How to test a neural network? Testing stages.
7. About training samples.

1. Can a profitable system be created using machine learning?

Probably, many beginners who have just started practicing real trading in the forex market without any particular system, took a sheet of paper and wrote a list of possibly suitable indicators, putting plus or minus sign next to them, or arrows, or price movement probability, based on the indicator chart in the terminal. Then the user would sum up observations and makes a certain decision to enter the market in a certain direction (or a decision of whether it is a good time to enter the market or better wait for another opportunity).

So, what happens in the most advanced neural network, i.e. our brain? After the observation of indicators, we have some image of a composite indicator generating the final signal, based on which we make a decision. Or a chain of signals is compiled into an indicator. Think about the following: if we study indicators at a certain point in time and look into the past for a maximum of several periods, how can we study these indicators simultaneously over several previous years and then compile a single composite indicator, which can further be optimized.

This is the answer to our second question: What can a neural network give us? Let's rephrase the question: what do you want to obtain form a neural network as a result of its training? Logically, the first question can also have an affirmative answer. This can be done programmatically. You can see how this is implemented in practice by watching my video: https://youtu.be/5GwhRnSqT78. You can also watch the video playlist featuring online testing of neural network modules at https://youtu.be/3wEMQOXJJNk

2. The rationale for training neural networks for decision making.

Before starting the development of any trading system, answer the following question: On what principles will this system function? We have two fundamental principles: trading flat and trend continuation. We will not consider derivatives from these two systems, such as intraday trading, use of fundamental data and news, trading at market opening time, etc. I came across descriptions of neural network products, in which authors suggested using them to forecast prices, such as stocks, currencies and so on.

1. Chart shows the operation of a neural network trained for price forecast

We can see that the neural network values repeat the price chart, but they are one step behind. The result does not depend on whether we predict price data or their derivatives. We can make some conclusion here. For example: "What is 'yesterday' for us is 'today' for the neural network." It is not quite useful, isn't it? However, this variant can be used after certain revision.

But our purpose is "What is 'today' for NN (neural network) is 'tomorrow' for us." Kind of a time machine. However, we understand that the best neural network is our brain. The efficiency of it is 50% (if we talk about the yes/no probability), or even worse. There is also the third option – "What is yesterday for the NN is today for us." Or: "What is today for us is yesterday for NN". Let us consider what the above situations mean in trading:

• First — we execute a deal and receive an answer from NN, whether the direction was right or not. However, we can know it without the NN.
• Second — we receive information from the NN, execute a deal and see the next day of the recommendation was correct.
• Third — we receive information from the NN about when to execute this or that deal, or if we need to execute a deal now - if we do, then in which direction.

The first variant is not suitable at all. The second and the third variants are quite suitable for trading. However, the second variant can be regarded as a glimpse into the future. Roughly speaking, a signal from the NN is received at a certain moment of time, for example at a day close - with a forecast of the next day closing level (and we are currently not interested in the price movement before the deal is closed). At this stage, this idea is hard to implement for a purely automated trading (for profitable trading). The idea of the third variant is that we track the response of the NN during the trading session. We interpret this response and either buy or sell an asset. Here, we need to understand the main thing.

The variant to be implemented depends on how we are going to train the neural network. In this case, the third variant is easier to implement. In the second variant, we use any information with the purpose of receiving the next-day result — its closure (the day is only used as an example, so this can be any period). In the third variant, we use the information received one step earlier, before we make a decision, showing where the price will move. I use the third variant in my systems.

3. Neural network: challenging or easy? 

We are trying to create a trading system. So, where do we take a neural network, how should we train it and how can we integrate it into the trading terminal? As for me, I am using ready-made neural networks: NeuroSolutions and Matlab. These platforms allow choosing a suitable network, train it and create an executable file with a desired interface. The resulting neural network program can look like this:

2. A neural network module created in Matlab environment

or like this:

3. A neural network module created using Neuro Solutions

When studying the possibilities of neural network application in financial markets, I came to the conclusion that neural networks can be used not only as the main signal generator, but also as an option for unloading the software part of the trading Expert Advisor. Imagine that you decide to write an Expert Advisor that uses a dozen indicators. These indicators have different parameters; they need to be analyzed and compared at some time period. Moreover, you use several time windows. Thus, you will receive an overloaded Expert Advisor for real trading, which is extremely hard to test.

What we can do is entrust the indicator calculation task to the neural network, after appropriate training. Further, the neural network will be trained using these indicators. It means that only the relative price data used in indicator formulas will need to be input to the neural network module from the Expert Advisor. The neural network will output "ones" and "zeros", which we can compare and make a decision.

Let us view the result using Stochastic Oscillator as an example. We will use the following price data as inputs. The indicator itself will be used as a training example.

4. Price Data

FileWrite(handle,
                   
                   iClose(NULL,0,i+4)-iLow(NULL,0,i+4),
                   iHigh(NULL,0,i+4)-iClose(NULL,0,i+4),
                   iHigh(NULL,0,i+4)-iLow(NULL,0,i+4),
                   iHigh(NULL,0,iHighest(NULL,0,MODE_HIGH,5,i+4))-iLow(NULL,0,iLowest(NULL,0,MODE_LOW,5,i+4)),
                   
                   iClose(NULL,0,i+3)-iLow(NULL,0,i+3),
                   iHigh(NULL,0,i+3)-iClose(NULL,0,i+3),
                   iHigh(NULL,0,i+3)-iLow(NULL,0,i+3),
                   iHigh(NULL,0,iHighest(NULL,0,MODE_HIGH,5,i+3))-iLow(NULL,0,iLowest(NULL,0,MODE_LOW,5,i+3)),
                   
                   iClose(NULL,0,i+2)-iLow(NULL,0,i+2),
                   iHigh(NULL,0,i+2)-iClose(NULL,0,i+2),
                   iHigh(NULL,0,i+2)-iLow(NULL,0,i+2),
                   iHigh(NULL,0,iHighest(NULL,0,MODE_HIGH,5,i+2))-iLow(NULL,0,iLowest(NULL,0,MODE_LOW,5,i+2)),
                   
                   iClose(NULL,0,i+1)-iLow(NULL,0,i+1),
                   iHigh(NULL,0,i+1)-iClose(NULL,0,i+1),
                   iHigh(NULL,0,i+1)-iLow(NULL,0,i+1),
                   iHigh(NULL,0,iHighest(NULL,0,MODE_HIGH,5,i+1))-iLow(NULL,0,iLowest(NULL,0,MODE_LOW,5,i+1)),
       
                   iClose(NULL,0,i)-iLow(NULL,0,i),
                   iHigh(NULL,0,i)-iClose(NULL,0,i),
                   iHigh(NULL,0,i)-iLow(NULL,0,i),
                   iHigh(NULL,0,iHighest(NULL,0,MODE_HIGH,5,i))-iLow(NULL,0,iLowest(NULL,0,MODE_LOW,5,i)),
 
                   iStochastic(NULL,0,5,3,3,MODE_SMA,1,MODE_MAIN,i),
                   TimeToStr(iTime(NULL,60,i)));

After training, the neural network will output the following result.

5. Neural network response

For a better visual study, let us move this data to the trading terminal as an indicator.

6. Stochastic and neural network indicator

The upper window shows the standard indicator available in the terminal. The lower window shows the indicator created by the neural network. We can see visually that the indicator created by the neural network has all the characteristics of a standard indicator, including levels, intersections, reversals, divergences, etc. Remember, we did not use any complex formulas to train the network.

Thus, we can draw the following block diagram of a trading system.

7. Trading system block diagram

MT4 blocks represent our Expert Advisor. "Input_mat" is the price file. "Open1,2,3" is the signal file. Examples of these files are provided in the next section.

The main work will concern the blocks "Net1" and "Net2". For these blocks, we will need to use several scripts and EAs to prepare historical data and to test signals from these blocks. Actually, when the system is ready as a complex, its modification, development and experimenting with it does not take much time. The following video shows an example: https://youtu.be/k_OLEKJCxPE. In general, preparation of files, training of Net1 and Net2, and the first testing stage in which we optimize the system, take 10 minutes.

4. Integrating a neural network into the trading terminal

Integration of a neural network and the trading terminal is not difficult. I solved this question by passing data via files created by the terminal and the neural network program. One may say that this may slow down decision making by the system. However, this method has its advantages. Firstly, the terminal passes a minimum of data, only a few tens of bytes. See below the file line written by the terminal.

8. File of normalized prices

Although this data transmission method enables deal opening only at the next tick after, following the arrival of a signal from the neural network. However, if the system does not trade ultra short-term moments, this is not crucial. In this article, the system works using open prices. Also, systems utilizing this data transmission method, requires testing by checkpoints or using the every tick mode. Tests of neural network-based systems in these two modes are almost identical. When developing traditional trading robots, I came across the situations when testing in the every tick mode showed much worse results. 

The main advantage of this data transmission mode is that we can control the data we receive and sent at every stage. I consider this to be one of the foundations for further successful trading using a neural network. Thus, our bulky preparation of the neural network system turns into an advantage in real work. This way we can reduce to a minimum the probability of receiving a program error in the system's logical structure. This is because the system requires a step-by-step triple testing before usage. We will get back to this part later.

The image below shows files “Input_mat” and “Bar”. These files are generated by the trading terminal. Files Open1,2,3 are generated by the NN program. The only inconvenience is that in the NN program we need to explicitly set the paths to these files based on how we use the EA - for testing or for trading.

9. Files generated by the neural network module and the Expert Advisor

“Bar” is an auxiliary file which is used as a counter.

The NN response is received into files Open1,2,3. The first line shows the previous response. The second line shows real-time response. This format is a special case. The format may differ depending on trading conditions. The number of response files may also be different. We have three of them, because the NN module uses three networks trained in different time intervals.

10. Neural network module response in files Open1,2,3

5. How to test a neural network? Testing stages

I use three testing stages when preparing trading systems based on neural networks. The first stage is rapid testing. This is the main system preparation stage in terms of its general performance. At this stage, we can optimize the system, while the optimization does not take much time. Here we use a script or an Expert Advisor for preparing a file with historical data, with the period following the historical period in which the NN was trained and up to the current time. Then, we receive NN responses at this interval, using a Matlab script, and create an indicator based on the responses. Then we use this indicator to optimize our NN responses for market entries and exits. The below figure shows an example of this indicator. This indicator is an interpretation of 52 derivatives from 12 custom indicators. These may include standard terminal indicators.

11. An indicator based on neural network responses

Next, we can optimize our trading strategy.

12. Results of testing neural network responses

In the second testing stage, we train and write neural networks in the Matlab environment using the Neural Network Toolbox.

13. Neural Fitting

14. Resulting neural networks

Get a response from these neural networks via the command window.

15. Receiving responses from neural networks

Thus, we will receive another indicator which should be identical to the previous one. Accordingly, testing of a strategy based on this indicator should also be identical.

If everything is good, we can move further.

We can test these neural networks using a script of the neural network module which will be used in the system. The signals should be tested at any time interval, using control points. If this test coincides with the identical time interval of the previous indicator test, then we are moving in the right direction. Launch this script in the Matlab environment. At the same time, launch the Expert Advisor in the trading terminal.

16. Launching a script in the Matlab environment

17. Launching the Expert Advisor in the terminal

Here is the result:

18. The result of testing the Matlab script and the MT4 Expert Advisor

Next, we need to create the user interface, compile the neural network module and test as is described above.

19. Testing the compiled neural network module

If the result is similar to the previous one, we can proceed to real trading using according to our neural network system.

5. About training samples

Depending on what training samples will be used to prepare the neural network, we will get different indicators based on NN responses. Therefore, different trading strategies can be created. Furthermore, a combination of different strategies will give us a more stable final result. One of the variants was shown in previous sections. In that example, we made a selection based on trading period extreme points. Let me give you another example.

20. Indicator of responses of a neural network trained on a different sample

In this case, I trained two neural networks. One - for buying, the other one - for selling. Training is performed on samples when the minimum price has been reached, while the maximum price has not yet been reached. And vice versa. These two indicators shown in the figure reflect the interpretation of twelve custom indicators. Highs of the red line show when the minimum price is reached. Gray highs are the maximum price. Now, it is possible to optimize these indicators, either separately or together. For example, we can test their intersections or difference in their values, as well as level intersections and so on.

It would be much more difficult to optimize twelve indicators.

Conclusion

There are many articles on the use of neural networks in trading. However, there is very little material regarding how to apply systems based on neural networks in practice. Furthermore, publications are intended for users with specific programming knowledge. It is quite difficult to provide a complete description in one article. I tried to explain the usage specifics without adding excessive theoretical material in my book "Neural network trading system. MetaTrader 4 + MATLAB. Step-by-step development. Second edition" (in Russian).