Making a Python trading system for MT. - page 3

New comment
 
Yuriy Asaulenko:

That's a different subject).

I don't put a team together. I walk on my own, do what I think is right and interesting, and make no commitments.

If anyone wants to go out with me, there's no objection and no commitment either.

I'm not talking about the team, but about the engine in the form of an EXE application to link Python and R with MQL, which I suggested in that thread.
 
Ivan Negreshniy:
I'm not talking about the team but the engine in the form of EXE application to link Python and R with MQL, which I suggested in that thread.

You'll get an EXE - no problem in Python. R, sorry, I don't use it - imho, a big dump of everything and anything. If there are diamonds in this dump, their search is problematic. Unless SanSanych tells me).

I deal mostly with neural networks, and there are enough of them without R.

But, actually, the methodology of interaction is planned to be universal, and probably you can use it for R as well.

 
Now that we have a pre-built application, it's time to exchange data between the terminal and Python.
For starters, we'll choose the simplest kind of exchange - data exchange via files.
Now a lot of people will snicker in their inkwells, and in vain. So let them).
1. During the construction and debugging of the application, file sharing is most efficient, as it requires no effort to upgrade. And once the application is built, it can be replaced by something else.
2. In the majority of ready-made TCs, file exchange speed is more than enough. 50-100 ms. Not enough for you?
3. When exchanging files via RAM-Disk exchange rate reaches 1.5 GByte/s. It's impossible to measure latency at all.
All in all, spit it out and forget the opponents.)
Well, actually, another exchange is planned for the last phase, if we or the topic live up to it).

We will exchange with the terminal via CSV files. So far one way - from the terminal to the application.
To avoid conflict of interest, we will use semaphore files.
So, let DataTS.csv be the data file, and STS.s be the semaphore file.
The terminal writes data into DataTS.csv, and, after writing, creates an empty semaphore file STS.s.
The application checks the presence of the STS.s semaphore file, then reads the DataTS.csv file, then deletes the STS.s semaphore file.
Then the terminal detects the disappearance of the STS.s semaphore file and only then writes the new data into the DataTS.csv file.
Possible variations, depending on Phyton and MT file functions - haven't looked yet.
However, not all at once, first we need to work with Python, and determine its capabilities.
That's what we'll do.
 
There have been some changes since my last visit. The app is now fully functional - everything described in the previous post has been implemented. All buttons now work and the file exchange is implemented.
- The Clear button clears the output window,
- The Send button sends the input window line to the output window,
- the Start button switches on file exchange,
- The Stop button stops file exchange.
This is easy to check. We create folder C:\PyTS, put data file DFile.csv and place semaphore file SFile.tx. Our Python program detects the semaphore file, reads the data file, displays it in the output window and deletes the semaphore file.
Now you can change the data in DFile.csv and create a new semaphore file, then the program will read and output the new data.
You can write a simple MQL-program, change the reading folder in our program to the folder where MT writes, and then you can independently estimate the exchange speed, possible data skips and many other things.
The only function responsible for file exchange is FileExchange(). It is implemented in an additional thread - the timer thread. We can have tens of such threads and functions and they will not affect the application in any way - i.e. we can have dozens of parallel and independent data exchange channels at our disposal.
Right now the timer is set to 10.0s - you can set it to 0.001s, see documentation.
Well, if you really want to measure speed and other characteristics - remove debug print() functions in exchange path - they significantly reduce speed.
The Python program code is in the attachment. As usual, rename it to .py
Files:
PyTS_1.0.txt  3 kb
 

We talked about exchanging CSV files, but version 1.0 (see previous post) just opens and reads files. The next version - 1.01, after some modifications, reads CSV files and stores them in InData variable.

Actually all modifications:

def FileExchange():
    global Change
    if Change==True:
        if os.path.exists(SFile):
            print("SFile.txt is")
            if os.path.exists(DFile):
                print("DFile.csv is")
                InData = csv.reader(open(DFile, 'r'))
                data_read = [row for row in InData]
                print(data_read)
                os.remove(SFile)
            else:
                print("DFile.csv no")
        else:
            print("SFile.txt no")
        print('Вывод таймера')
        timer =threading.Timer(10.0,FileExchange)
        timer.start()

Only 3 lines changed + csv library connection.

And also control output of CSV contents from InData variable:

In [18]: [['04.08.2018 14:00', ' 1300', ' 1295', '1310', '1305', '100'], ['04.08.2018 14:10', ' 1307', ' 1297', '1320', '1315', '150']]

Now the values of rows are available by indexes, type - row, column.

Well, and the code itself - see attachment.

Files:
PyTS_1.01.txt  3 kb
 
Yuriy Asaulenko:

We talked about exchanging CSV files, but version 1.0 (see previous post) just opens and reads files. The next version - 1.01, after some modifications, reads CSV files and stores them in InData variable.

Actually all modifications:

Only 3 lines changed + csv library connection.

And also control output of CSV contents from InData variable:

Now the values of rows are available by indexes, type - row, column.

Well, and the code itself - see attachment.

Using file sharing is not the best solution. File operations are very slow. You have to connect a RAM disk for such an exchange.
 
Grigoriy Chaunin:
Using files for exchange is not the best solution. File operations are very slow. You have to connect a RAM disk for such an exchange.
Can you give the performance characteristics of the file exchange (what is there) and the tasks required (what is needed)?


And I, for one, will say that they are extremely fast, not slow. Without numerical characteristics, both your and my statements are based on nothing, and therefore make no sense in the task at hand. You need characteristics of both the file exchange and the needs of the task, and only after comparing them can you come to valid conclusions. Since Yuri Asaulenko decided that files are suitable, I think he knows better the planned volume of data transfer and processing time.

In general, disk files have a great advantage over volatile memory - they are stored on the shutdown disk and can be accessed after a week or a year. To control, to analyse, to verify, just to observe.

 
Grigoriy Chaunin:
Using files to swap is not the best solution. File operations are very slow. You have to connect a RAM disk for such an exchange.

This issue has already been discussed in this thread, in previous posts, including the possible use of RAM-Disk, if necessary:

Forum on trading, automated trading systems and testing trading strategies

How to make trading system for MT using Python.

Yuriy Asaulenko, 2018.08.01 19:33

Now that we have the blueprint of the application, it's time to engage in data exchange of the terminal with Python.
Firstly we'll choose the simplest type of exchange - data exchange via files.
Now many people will snigger in their inkwells, and in vain. So let them).
1. During the construction and debugging of the application, file sharing is most efficient, as it requires no effort to upgrade. And once the application is built, it can be replaced by something else.
2. In the majority of ready-made TCs, file exchange speed is more than enough. 50-100 ms. Not enough for you?
3. When exchanging files via RAM-Disk exchange rate reaches 1.5 GByte/s. It's impossible to measure latency at all.
All in all, spit it out and forget the opponents.)
Well, actually, another exchange is planned for the last phase, if we or the topic live up to it).
Also in one of the first posts of the thread.
 

I present a new version of PyTS 1.02. This version is functionally equivalent to version 1.01, but the unnecessary print() controls were removed from the code and the class approach began to be implemented (Lenin wrote - the approach must be class based) - CSV file processing is fully assigned to the class - cCSVJob.

Zip-file in the attachment, where you will also find all previous versions of the program, as well as, in the PyTS folder, necessary files with source data for testing.

Files:
PyTS.zip  6 kb
12345678910...17
New comment
Reason: