MT5-Report via Python

 

Hi, 

I'm using mostly the code from  history_deals_get - Python Integration - MQL5 Reference - Reference on algorithmic/automated trading language for MetaTrader 5
and it works: https://youtu.be/9gXsOIand1s

But not perfectly, I say:

...
..
from_date=datetime(2023,5,1)
to_date=datetime(2023,5,3)
#to_date=datetime.now()


# get deals for symbols whose names contain neither "EUR"
deals = mt5.history_deals_get(from_date, to_date, group="*,!*EUR*")
...
..

and get what I want:

                  time symbol  volume     price  profit  points  cumulative_profit  cumulative_points
0  2023-05-01 11:36:15  USTEC     0.1  13228.50    0.00     0.0               0.00                0.0
1  2023-05-01 11:45:14  USTEC     0.1  13229.70    0.11     1.1               0.11                1.1
2  2023-05-01 15:15:19  USTEC     0.1  13217.10    0.00     0.0               0.11                1.1
3  2023-05-01 15:47:17  USTEC     0.1  13226.60    0.86     8.6               0.97                9.7
4  2023-05-01 18:16:27  USTEC     0.1  13201.80    0.00     0.0               0.97                9.7
5  2023-05-01 18:21:41  USTEC     0.1  13207.80    0.55     5.5               1.52               15.2
6  2023-05-01 19:30:50  USTEC     0.1  13252.50    0.00     0.0               1.52               15.2
7  2023-05-01 20:13:48  USTEC     0.1  13278.30    0.00     0.0               1.52               15.2
8  2023-05-01 20:42:20  USTEC     0.1  13284.70    0.00     0.0               1.52               15.2
9  2023-05-01 21:01:11  USTEC     0.3  13270.40    0.39     1.3               1.91               16.5
10 2023-05-01 21:12:42  USTEC     0.1  13269.10    0.00     0.0               1.91               16.5
11 2023-05-01 21:16:30  USTEC     0.1  13259.10    0.00     0.0               1.91               16.5
12 2023-05-02 10:02:38   DE40     0.1  15959.44    0.00     0.0               1.91               16.5
13 2023-05-02 10:22:31   DE40     0.1  15940.44    1.90    19.0               3.81               35.5
14 2023-05-02 10:33:11   DE40     0.1  15919.44    0.00     0.0               3.81               35.5
15 2023-05-02 11:00:34   DE40     0.1  15857.94    0.00     0.0               3.81               35.5
16 2023-05-02 12:40:17   DE40     0.2  15890.67    0.40     2.0               4.21               37.5
17 2023-05-02 17:01:25   DE40     0.1  15833.51    0.00     0.0               4.21               37.5
18 2023-05-02 17:20:25   DE40     0.1  15765.80   -6.77   -67.7              -2.56              -30.2
19 2023-05-02 17:35:52  USTEC     0.1  13107.90    0.00     0.0              -2.56              -30.2
20 2023-05-02 17:36:10  USTEC     0.1  13104.00    0.00     0.0              -2.56              -30.2
21 2023-05-02 17:37:11  USTEC     0.1  13094.80    0.00     0.0              -2.56              -30.2

Then I say:

..
...
from_date=datetime(2023,5,1)
to_date=datetime(2023,5,5)
#to_date=datetime.now()

# get deals for symbols whose names contain neither "EUR"
deals = mt5.history_deals_get(from_date, to_date, group="*,!*EUR*")
...
..

and I get no data.
As one can see in the video above there are two days missing in the last dataframe; all deals were received, they are in the original dataframe and in my first filtered one, everything there, but when I add my two columns it just stopps printing in the middle.. of even returns nothing.. depending on the days chosen.. it's not a problem with the broker as all is received.. and shouldn't be a problem with python as it works with a day less and pandas is well qa'ed an widely used..

I have this set:

pd.set_option('display.max_rows', None)
pd.options.display.width = None


Thanks =)

Documentation on MQL5: Python Integration / history_deals_get
Documentation on MQL5: Python Integration / history_deals_get
  • www.mql5.com
history_deals_get - Python Integration - MQL5 Reference - Reference on algorithmic/automated trading language for MetaTrader 5
 
Christian Linden:

Hi, 

I'm using mostly the code from  history_deals_get - Python Integration - MQL5 Reference - Reference on algorithmic/automated trading language for MetaTrader 5
and it works: https://youtu.be/9gXsOIand1s

But not perfectly, I say:

and get what I want:

Then I say:

and I get no data.
As one can see in the video above there are two days missing in the last dataframe; all deals were received, they are in the original dataframe and in my first filtered one, everything there, but when I add my two columns it just stopps printing in the middle.. of even returns nothing.. depending on the days chosen.. it's not a problem with the broker as all is received.. and shouldn't be a problem with python as it works with a day less and pandas is well qa'ed an widely used..

I have this set:


Thanks =)

What do you mean by adding "two columns" ?

 

I loop through the original df (from:   history_deals_get - Python Integration - MQL5 Reference - Reference on algorithmic/automated trading language for MetaTrader 5)

to filter my own "df_points":

df_points = pd.DataFrame(columns=('time','symbol','volume','price','profit','points'))
for pos, d in df.iterrows():
    points = d.profit / d.volume
    df_points.loc[pos] = [d.time,d.symbol,d.volume,d.price,d.profit,points]

print(df_points)

and then I add two columns by:

df_points = df_points.astype({"points": float})
df_points['cumulative_profit'] = df_points['profit'].cumsum().round(2)
df_points = df_points.astype({"cumulative_profit": float})
df_points['cumulative_points'] = df_points['points'].cumsum().round(2)
df_points = df_points.astype({"cumulative_points": float})

print(df_points)

As mentioned this works for many "from-tos" but at some it just stopps before the end.

Here is the full code:

import MetaTrader5 as mt5
import pandas as pd
import pytz
from datetime import datetime

pd.set_option('display.max_rows', None)
pd.options.display.width = None

# establish connection to MetaTrader 5 terminal
if not mt5.initialize():
    print("initialize() failed, error code =",mt5.last_error())
    quit()

account = 12345678 # your account
authorized = mt5.login(account, password="yourPassword", server="yourBrokerServer")
if authorized:
    account_info_dict = mt5.account_info()._asdict()
    for prop in account_info_dict:
        print("   {}={}".format(prop, account_info_dict[prop]))
else:
    print("failed to connect at account #{}, error code: {}".format(account, mt5.last_error()))


# get the number of deals in history
from_date=datetime(2023,3,6)
to_date=datetime(2023,3,10)
#to_date=datetime.now()

# get deals for all symbols except EUR
deals = mt5.history_deals_get(from_date, to_date, group="*,!*EUR*")
if deals == None:
    print("No deals, error code={}".format(mt5.last_error()))
elif len(deals) > 0:
    print("history_deals_get(from_date, to_date, group=\"*,!*EUR*,\") =", len(deals))
    # display all obtained deals 'as is'
    for deal in deals:
        print("  ",deal)
    print()
    # display these deals as a table using pandas.DataFrame
    df=pd.DataFrame(list(deals),columns=deals[0]._asdict().keys())
    df['time'] = pd.to_datetime(df['time'], unit='s')
    print(df)
print("")


# defining new dataframe including the points win/loss per trade
df_points = pd.DataFrame(columns=('time','symbol','volume','price','profit','points'))
for pos, d in df.iterrows():
    points = d.profit / d.volume
    df_points.loc[pos] = [d.time,d.symbol,d.volume,d.price,d.profit,points]

print(df_points)
print("")

# adding column to dataframe to get the cumulated points after each trade
df_points = df_points.astype({"points": float})
df_points['cumulative_points'] = df_points['points'].cumsum().round(2)
df_points = df_points.astype({"cumulative_points": float})
print(df_points)

print("")

mt5.shutdown()

Here you have the problem: 6.3-10.3  and my df_points ends at 7.3 =(

I just see that the original df ends at 8.3. Why? I can't see it from the deals but probably to all are fetched, maybe there's a limit?

But if df contains deals until 8.3 I can't see any reason why my df_points ends at 7.3.

Documentation on MQL5: Python Integration / history_deals_get
Documentation on MQL5: Python Integration / history_deals_get
  • www.mql5.com
history_deals_get - Python Integration - MQL5 Reference - Reference on algorithmic/automated trading language for MetaTrader 5
 
Christian Linden #:

I loop through the original df (from:   history_deals_get - Python Integration - MQL5 Reference - Reference on algorithmic/automated trading language for MetaTrader 5)

to filter my own "df_points":

and then I add two columns by:

As mentioned this works for many "from-tos" but at some it just stopps before the end.

Here is the full code:

Here you have the problem: 6.3-10.3  and my df_points ends at 7.3 =(

I just see that the original df ends at 8.3. Why? I can't see it from the deals but probably to all are fetched, maybe there's a limit?

But if df contains deals until 8.3 I can't see any reason why my df_points ends at 7.3.

I found it =)

If there's a divident when you hold the position over night the volume is 0 and then

for pos, d in df.iterrows():
    points = d.profit / d.volume

makes a problem called "devision by zero" =)

The vscode didn't show me the error.

I expanded to:

for pos, d in df.iterrows():
    if d.volume > 0:
        points = d.profit / d.volume
        df_points.loc[pos] = [d.time,d.symbol,d.volume,d.price,d.profit,points]

and it seems to work fine as I get my whole last week:

                   time symbol  volume     price  profit     points  cumulative_profit  cumulative_points
0   2023-05-01 11:36:15  USTEC     0.1  13228.50    0.00   0.000000               0.00               0.00
1   2023-05-01 11:45:14  USTEC     0.1  13229.70    0.11   1.100000               0.11               1.10
2   2023-05-01 15:15:19  USTEC     0.1  13217.10    0.00   0.000000               0.11               1.10
3   2023-05-01 15:47:17  USTEC     0.1  13226.60    0.86   8.600000               0.97               9.70
4   2023-05-01 18:16:27  USTEC     0.1  13201.80    0.00   0.000000               0.97               9.70
5   2023-05-01 18:21:41  USTEC     0.1  13207.80    0.55   5.500000               1.52              15.20
6   2023-05-01 19:30:50  USTEC     0.1  13252.50    0.00   0.000000               1.52              15.20
7   2023-05-01 20:13:48  USTEC     0.1  13278.30    0.00   0.000000               1.52              15.20
8   2023-05-01 20:42:20  USTEC     0.1  13284.70    0.00   0.000000               1.52              15.20
9   2023-05-01 21:01:11  USTEC     0.3  13270.40    0.39   1.300000               1.91              16.50
10  2023-05-01 21:12:42  USTEC     0.1  13269.10    0.00   0.000000               1.91              16.50
11  2023-05-01 21:16:30  USTEC     0.1  13259.10    0.00   0.000000               1.91              16.50
12  2023-05-02 10:02:38   DE40     0.1  15959.44    0.00   0.000000               1.91              16.50
13  2023-05-02 10:22:31   DE40     0.1  15940.44    1.90  19.000000               3.81              35.50
14  2023-05-02 10:33:11   DE40     0.1  15919.44    0.00   0.000000               3.81              35.50
15  2023-05-02 11:00:34   DE40     0.1  15857.94    0.00   0.000000               3.81              35.50
16  2023-05-02 12:40:17   DE40     0.2  15890.67    0.40   2.000000               4.21              37.50
17  2023-05-02 17:01:25   DE40     0.1  15833.51    0.00   0.000000               4.21              37.50
18  2023-05-02 17:20:25   DE40     0.1  15765.80   -6.77 -67.700000              -2.56             -30.20
19  2023-05-02 17:35:52  USTEC     0.1  13107.90    0.00   0.000000              -2.56             -30.20
20  2023-05-02 17:36:10  USTEC     0.1  13104.00    0.00   0.000000              -2.56             -30.20
21  2023-05-02 17:37:11  USTEC     0.1  13094.80    0.00   0.000000              -2.56             -30.20
22  2023-05-03 14:50:27  USTEC     0.4  13139.00  -10.14 -25.350000             -12.70             -55.55
23  2023-05-03 16:30:06  USTEC     0.1  13123.80    0.00   0.000000             -12.70             -55.55
24  2023-05-03 16:45:09  USTEC     0.2  13146.40    0.18   0.900000             -12.52             -54.65
25  2023-05-03 17:11:52  USTEC     0.1  13128.10    0.00   0.000000             -12.52             -54.65
26  2023-05-03 17:13:56  USTEC     0.1  13132.60    0.41   4.100000             -12.11             -50.55
27  2023-05-03 17:29:32  USTEC     0.1  13126.80    0.00   0.000000             -12.11             -50.55
28  2023-05-03 17:35:00  USTEC     0.1  13137.80    1.00  10.000000             -11.11             -40.55
29  2023-05-03 18:01:36  USTEC     0.1  13125.50    0.00   0.000000             -11.11             -40.55
30  2023-05-03 18:58:56  USTEC     0.1  13138.50    1.18  11.800000              -9.93             -28.75
31  2023-05-03 21:23:03  USTEC     0.1  13199.10    0.00   0.000000              -9.93             -28.75
32  2023-05-03 21:30:47  USTEC     0.1  13192.70    0.58   5.800000              -9.35             -22.95
33  2023-05-03 21:40:20  USTEC     0.1  13215.20    0.00   0.000000              -9.35             -22.95
34  2023-05-03 21:41:46  USTEC     0.1  13209.20    0.54   5.400000              -8.81             -17.55
35  2023-05-03 22:09:13  USTEC     0.1  13112.80    0.00   0.000000              -8.81             -17.55
36  2023-05-03 22:09:44  USTEC     0.1  13116.80    0.36   3.600000              -8.45             -13.95
37  2023-05-03 22:09:55  USTEC     0.1  13125.10    0.00   0.000000              -8.45             -13.95
38  2023-05-03 22:10:57  USTEC     0.1  13089.60    0.00   0.000000              -8.45             -13.95
39  2023-05-03 22:12:09  USTEC     0.2  13107.60    0.05   0.250000              -8.40             -13.70
40  2023-05-03 22:13:00  USTEC     0.1  13085.40    0.00   0.000000              -8.40             -13.70
41  2023-05-03 22:13:54  USTEC     0.1  13099.40    1.27  12.700000              -7.13              -1.00
42  2023-05-03 22:15:06  USTEC     0.1  13083.10    0.00   0.000000              -7.13              -1.00
43  2023-05-03 22:16:20  USTEC     0.1  13088.30    0.47   4.700000              -6.66               3.70
44  2023-05-03 22:20:12  USTEC     0.1  13081.30    0.00   0.000000              -6.66               3.70
45  2023-05-03 22:26:01  USTEC     0.1  13106.70    2.30  23.000000              -4.36              26.70
46  2023-05-03 22:27:51  USTEC     0.1  13105.10    0.00   0.000000              -4.36              26.70
47  2023-05-03 22:29:09  USTEC     0.1  13058.40    0.00   0.000000              -4.36              26.70
48  2023-05-03 22:31:57  USTEC     0.1  13047.10    0.00   0.000000              -4.36              26.70
49  2023-05-03 22:32:18  USTEC     0.1  13044.10    0.00   0.000000              -4.36              26.70
51  2023-05-04 05:49:06  USTEC     0.4  13075.30    4.20  10.500000              -0.16              37.20
52  2023-05-04 09:12:09   DE40     0.1  15807.77    0.00   0.000000              -0.16              37.20
53  2023-05-04 10:06:18   DE40     0.1  15727.82    0.00   0.000000              -0.16              37.20
54  2023-05-04 10:11:48   DE40     0.1  15695.82    0.00   0.000000              -0.16              37.20
55  2023-05-04 10:37:44   DE40     0.3  15763.45    5.89  19.633333               5.73              56.83
56  2023-05-04 10:50:01   DE40     0.1  15754.22    0.00   0.000000               5.73              56.83
57  2023-05-04 10:57:42   DE40     0.1  15786.22    3.20  32.000000               8.93              88.83
58  2023-05-04 11:17:48   DE40     0.1  15784.72    0.00   0.000000               8.93              88.83
59  2023-05-04 11:24:12   DE40     0.1  15781.22    0.35   3.500000               9.28              92.33
60  2023-05-04 11:55:18   DE40     0.1  15755.72    0.00   0.000000               9.28              92.33
61  2023-05-04 13:04:51   DE40     0.1  15695.22    0.00   0.000000               9.28              92.33
62  2023-05-04 15:02:02   DE40     0.1  15668.89    0.00   0.000000               9.28              92.33
63  2023-05-04 15:18:05   DE40     0.1  15661.39    0.00   0.000000               9.28              92.33
64  2023-05-04 15:30:34   DE40     0.4  15708.89    5.43  13.575000              14.71             105.91
65  2023-05-04 15:30:53  USTEC     0.1  13014.70    0.00   0.000000              14.71             105.91
66  2023-05-04 16:32:41  USTEC     0.1  12982.50    0.00   0.000000              14.71             105.91
..

...  too many character (>64999)
..
78  2023-05-04 20:26:38  USTEC     0.1  13035.00    0.65   6.500000              23.54             168.51
79  2023-05-04 20:26:53  USTEC     0.1  13029.20    0.00   0.000000              23.54             168.51
80  2023-05-04 20:28:46  USTEC     0.1  13043.70    1.31  13.100000              24.85             181.61
81  2023-05-04 20:32:50  USTEC     0.1  13057.00    0.00   0.000000              24.85             181.61
82  2023-05-04 20:38:40  USTEC     0.1  13050.20    0.62   6.200000              25.47             187.81
83  2023-05-04 20:46:39  USTEC     0.1  13039.00    0.00   0.000000              25.47             187.81
85  2023-05-05 03:49:48  USTEC     0.1  13042.80    0.34   3.400000              25.81             191.21
86  2023-05-05 10:01:15   DE40     0.1  15821.65    0.00   0.000000              25.81             191.21
87  2023-05-05 10:04:47   DE40     0.1  15815.65    0.60   6.000000              26.41             197.21
88  2023-05-05 10:05:29   DE40     0.1  15830.15    0.00   0.000000              26.41             197.21
89  2023-05-05 10:06:17   DE40     0.1  15825.15    0.50   5.000000              26.91             202.21
90  2023-05-05 10:25:29   DE40     0.1  15833.65    0.00   0.000000              26.91             202.21
91  2023-05-05 10:26:14   DE40     0.1  15829.65    0.40   4.000000              27.31             206.21
92  2023-05-05 10:28:02   DE40     0.1  15823.15    0.00   0.000000              27.31             206.21
93  2023-05-05 10:28:22   DE40     0.1  15819.65    0.35   3.500000              27.66             209.71
94  2023-05-05 10:59:41   DE40     0.1  15783.65    0.00   0.000000              27.66             209.71
95  2023-05-05 11:45:46   DE40     0.1  15788.91    0.53   5.300000              28.19             215.01
96  2023-05-05 11:47:23   DE40     0.1  15793.91    0.00   0.000000              28.19             215.01
97  2023-05-05 12:02:05   DE40     0.1  15848.91    0.00   0.000000              28.19             215.01
98  2023-05-05 20:35:46  USTEC     0.1  13232.40    0.00   0.000000              28.19             215.01
99  2023-05-05 21:35:45  USTEC     0.1  13260.50   -2.55 -25.500000              25.64             189.51
100 2023-05-05 21:58:01   DE40     0.1  16003.86    0.00   0.000000              25.64             189.51

I'm not clean with it at all as it stopped at different interruptions w/o any error but actually I'm fine with it as I get my points I need.

Thank you!

Reason: