Making a Python trading system for MT. - page 4

 

Everything is ready now, and it's time to link our Python TS to the MT terminal. However, there are some actions to be taken in order to do this.

Our future TC reads and will write files to the folder C:\PyTS, but the MT terminal, at some unknown whim of MQ, can read-write files only to its own directory - C:\<much letters and numbers>\Files, and no other way. Of course, you can change settings in Python code, and let it swap to native MQ folder, but I need terminal to read-write files from there and to where I want it, not to where it, the terminal, wants to. The more so, it will be necessary in future.

Actually, this question was solved a couple of years ago, in my topic - https://www.mql5.com/ru/forum/79922.

Everything is solved with Windows tools. In terminal folder C:\<a lot of letters and numbers> \Files you put reference to folder C:\PyTS, that is interpreted by terminal (and OS too) as subfolder Files\PyTS and terminal writes there, in C:\PyTS, as in its native one).

This is done at the command line, using the native Windows MKLINK utility. For those who can't do it themselves, here is a detailed description:

Type in execute - cmd, or from the menu - Utilities-Windows -> Command Prompt. You get a window where you type MKLINK or mklink. Here we have:


Now type cd... several times until you get to C:\.

Then open Windows Explorer, and go to C:\<a lot of letters and numbers>\Files, where terminal puts files, and select and copy path to folder Ctrl-Ins.

Go back to the command prompt window and there write cd <space>, use Shift-Ins to paste in the path to ... \Files, and Enter. We get it:

Now in the command line we write - mklink /D PyTS c:\PyTS , and as usual - Enter. Here we have it:

We see that a link has been created. We see that it appeared in the folder ......\Files and we access it - we see the files in the folder C:\PyTS. )) That's it.

Now, all files written by terminal to folder ...Files\PyTS will, in fact, be written to folder C:\PyTS.

.

RAM Диск.
RAM Диск.
  • 2016.04.07
  • www.mql5.com
Общее обсуждение: RAM Диск.
 
So, we're done with the file sharing. Everything works, all sources are in the attachment. Everything else, and the reverse transfer from Python to MT, is done the same way, and I hope will not cause any problems. To run, read and execute my previous post. There is no other way.)

The MQL code is also in the attachment. - MQL reads the history, and passes it to the Python application. Python accepts it and prints it. In Python don't forget to press the Start button. )

I hope there's no need to write comments on MQL code - there's a reference). Yes, data transfer from MT is done on timer, it is more convenient for me, at this stage.

If you have any questions, do not hesitate to call).

I will pause for now - now we need to solve questions with the strategy. If you have any suggestions - we'll consider).

Once again, file sharing is ideal for strategy development and debugging, as it requires no effort to modify. For most real-world strategies, it is still suitable. But, if needed, we can always replace it with something else, without any significant changes to the program - a matter of 1-2 days.

Files:
PyTS.zip  4 kb
PyTSMT.mq5  7 kb
 

I don't promise to write often, only as I get it. The project is being done in the background, with little to no time to spare.

Thought it would be good for the system to display graphs

In general, Python is a simple language - a couple of days to study it is enough. But to deal with modules is much harder - there are thousands of them, and who does what and why is not always obvious. With graphics, that's what happened - I had to sort it out for a long time. But the result is as follows:

The program reads a CSV file and draws a chart.

The chart shows prices by Close, about 50 thousand candlesticks.

If you click on the magnifying glass and select a rectangle in the chart area like this:

you can zoom in on the chart, and you'll get it:

This can be done repeatedly. The chart can also be moved and saved as a picture, and of course expanded to full screen.

All the files you need are in the application. Unzip zip to some folder and run CSVGraph.py

Before running it, don't forget to check if the modules you want to import are installed.

Files:
CSVGraph.zip  582 kb
 
As an exercise in Python, perhaps. But from a practical point of view, it is of little use + a waste of time. Of course, everyone has their own way. Good luck
 

Let's leave the nonsense to the neighbours and amateur sounding board, but let's sum up the preliminary results ourselves.

If you're already familiar with the subject If you're a pro, you already know some Python, you know how to work with CSV files, you know how to build charts and exchange data with the terminal. This is already enough to develop, simulate and test trading strategies. If your strategy works with TF 1 min and longer without using ticks and accesses the terminal with 5-10 sec intervals, you may already create working strategies directly in Phyton, leaving only delivery of data and "execution" of your orders to the terminal.

Besides, you already know programming and the Phyton environment, at a level beyond the language and environment R, so popular among the local apologists of machine learning. You will already be able to build even very complex indicator strategies in Python, but all features of Python (similarly to R) can be implemented using modules. In this topic we have used only 3 modules - tkinter, numpy and matplotlib.

Now, what Python needs to build more advanced strategies, say using machine learning. The -scikit-learn module can be used for this. With the use ofscikit-learn the following features become available - User's Guide. How do you like the set of methods? - It already has almost everything, including, not unfamiliar from R and the Machine Learning topic, from Bayesian classification to random forests and neural networks.

This is just one of the machine learning modules in Python, and there are about a dozen of them. Thescikit-learn module is said to not be the best of them, nor the most advanced. That depends on who needs what. The others are not hard to find on the internet.


PS For those who are starting or just getting started with Python.

I've tried several Python development environments and stopped at the most convenient for beginners (which include myself) - Anaconda. What makes it convenient is that everything I need is already installed, including Python. Editor I use Spyder - installed withAnaconda. I also advise you to install editor VSCode - also very nice and installed directly fromAnaconda.

You might not be happy with my choice.

 
After 4 pages of the topic, we are finally ready to move on to what has been stated - strategies in Python. Everything is already done for this.
We will start with a simple strategy - trading on EMA crossing, possibly with modifications.
When I mastered neural networks (NS), I started with a simple task - the NS should identify points of EMA crossover. The task, from a practical point of view, is absolutely useless, but it allowed us to determine how to feed the NS with market data, how to prepare it, and solved many problems of training.
Now our task is not to make profit at all and not even to link it with the terminal - it is simple, but without a profitable strategy it is meaningless. Our task now is to create a strategy template, and learn how to test it, and any strategy will do. Then put any strategy in the template and test it, and if it works, then we can start trading via the terminal.

The required indicators are all ready. See figure made in Python (In Python it is very convenient and fast to draw all sorts of charts). This is the response to a single jump - 1(t), or transient function. Actually, one of the standard tests.


The fm is the conventional moving average (serving only for calibration), f1 is the EMA, with a modified coefficient calculation, and f2 is a non-standard indicator.

However, as our mutual acquaintance used to say:

Igor Makanu:

I've said it a million times, and I'll probably say it again... Nobody has invented anything better than indicators from MT supply (standard... so to say old Slavic indicators... pagan!!!).

So, in order not to fool your head with science, you will get the standard EMA and strategy sources. However, I will make and test the strategy on my indicators. Maybe, I will hide something else from prying eyes, but without loss of functionality.

But it is not for the EMA crossing that this is all about. The goal of all this is machine learning strategies. So far I'm testing different classification methods. And here are the test results:

Gaussian, RBF SVM, Neural Networks and naive Bayesian classification are at the top so far. The Python program is already ready, available somewhere on the internet, but the data preparation is up to the user.

Let's go.

 
Yuriy Asaulenko:
After 4 pages of the topic, we are finally ready to move on to what has been stated - strategies in Python. Everything is already done for this.
We will start with a simple strategy - trading on EMA crossing, possibly with modifications.
When I mastered neural networks (NS), I started with a simple task - the NS should identify points of EMA crossover. The task, from a practical point of view, is absolutely useless, but it allowed us to determine how to feed the NS with market data, how to prepare it and solve many training issues.
Now our task is not to make profit at all and not even to communicate with the terminal - it is simple, but without a profitable strategy it is meaningless. Our task now is to create a strategy template, and learn how to test it, and any strategy will do. Then put any strategy into the template and test it, and if it works, then we can start trading via the terminal.

The required indicators are all ready. See figure made in Python (In Python it is very convenient and fast to draw all sorts of charts). This is the response to a single jump - 1(t), or the transition function. Actually, one of the standard tests.


The fm is the conventional moving average (serving only for calibration), f1 is the EMA, with a modified coefficient calculation, and f2 is a non-standard indicator.

However, as our mutual acquaintance used to say:

So, to avoid fooling your head with science, you will get the standard EMA and the sources of the strategy. However, I will make and test the strategy on my indicators. Maybe, I will hide something else from prying eyes, but without loss of functionality.

But it is not for the EMA crossing that this is all about. The goal of all this is machine learning strategies. So far I'm testing different classification methods. And here are the test results:

Gaussian, RBF SVM, Neural Networks and naive Bayesian classification are at the top so far. The Python program is already ready, available somewhere on the internet, but the data preparation is up to the user.

Let's go.


It's a shame they switched to neural networks. But mostly very useful, I think it will come in handy.

Forum on trading, automated trading systems & strategy testing

Machine Learning Robots

Yuriy Asaulenko, 2018.09.21 00:45

They will not. The maximum they will do is the same as before.


 
Sergey Chalyshev:


It's a pity they switched to neural networks. But mostly very useful, I think it will come in handy.

There are no plans to switch to NS and other MoD methods in this thread. For now, at least). We'll see.

What is being done here is a Python system template. Specific filling of this template is a matter of specific user.

Well, to show the capabilities of Python libraries, I think, is not superfluous.

 
PS Yes, I didn't say, I will do all the strategies for FORTS (MOEX). I don't play with forex OCs.
 
Let's continue our games. As of today, we have ready and functioning:
2. Strategy Template.
3. A simple EMA crossing strategy is placed in the template.

When the programme is started, the strategy is tested, the report of all trades is saved in CSV file, the profit graph is displayed on the console, which can be saved on the disk - see Fig.


The testing was done using SBER MOEX futures, 3M interval, 1M timeframe, total of ~55000 candles. Only close candlesticks were used in the strategy. The duration of testing is ~1 min, starting from the program start. This is about 1ms per candle.
A folder with all the necessary program files, including the history for SBER futures is attached.
To run the program - run the main.py file. Before running, make sure that your Python has all the necessary packages and modules installed. If you are running in Anaconda, all the required packages-modules are already installed.
Basically, write your strategy into the template, and test it out.
Now, how do I imagine the application of this technology.
It was decided to translate on Python platform proven strategy that uses neural networks, perhaps with a small upgrade. With this it will be possible with minimal expenses to master application of ML technologies available in Python, connect the system to the terminal and start direct operation. For me the main trading platform is MOEX, and Forex is auxiliary, so in the foreseeable future connection of the system to the MT terminal is not planned, although it was assumed at the beginning of the topic as an option for testing.
However, there are enough experts on the forum who, if they want, can do it. Moreover, C API Python is much easier than C API of the same R, which local specialists have successfully coped with.
We can consider that the objectives of the topic are completely fulfilled.
Files:
Public.zip  683 kb
Reason: