How to Export and Format MQL5 Indicator Data and Trade History for Python to Train a Machine Learning Model?

[Hydra Corporation]

Hello, community!

I am developing a project to integrate my trading environment in MetaTrader 5 with Python, aiming to train a Machine Learning model that can optimize my trading strategy. I already have my trade history and a custom indicator in MQL5. I would like to collect this data and prepare it for model training.

I would appreciate guidance on the following points:

1. Exporting MQL5 Indicator Data to Python

• How can I export the values generated by my custom indicator to be used in Python?
  • Is there a specific function or method in MQL5 that facilitates exporting this data to a file (such as CSV) or to a database?

2. Formatting Trade History for Training

• What is the best way to format my trade history to train a Machine Learning model?
  • What fields and structuring are recommended to ensure the data is suitable for analysis and modeling?
  • Examples of data structures (like tables) that facilitate the training process.

3. Machine Learning Model Recommendations

• Which models do you recommend for analyzing and predicting outcomes based on trade history?
  • Classification models (e.g., Random Forest, XGBoost) or regression models (e.g., Linear Regression, Gradient Boosting)?
  • Considerations for time series models, such as LSTM or ARIMA, in the context of financial data.

4. Integration Between MQL5 and Python

• Which Python libraries are best suited for reading and processing the data exported from MQL5?
  • Suggestions for frameworks or APIs that facilitate direct communication between MetaTrader 5 and Python for real-time data updates.

5. Best Practices in Data Preparation and Model Training

• How to prepare the collected data for effective Machine Learning model training?
  • Techniques for data cleaning, normalization, feature engineering, and feature selection.
  • Methods to avoid overfitting and ensure proper model validation.

6. Resources and Code Examples

• Are there any code examples or resources that can help in this integration and training process?
  • Links to tutorials, GitHub projects, or relevant documentation that facilitate development.

Additional Context:

• Final Objective: I want the Machine Learning model to assist in making trading decisions by identifying patterns that can improve the performance of my trades.
• Available Data: Detailed history of executed trades, including date/time, symbol, order type, volume, entry/exit price, stop loss, take profit, result, and custom indicator values.

Thank you in advance for your help and for any resources, code examples, or documentation you can share to facilitate this integration and model development process.

Best regards,

Hydra

[[Deleted]]

We cannot possibly answer all of your queries in just a few lines of text.

So, please put in the effort and time, and read the documentation and the books, so that you can learn to do all the things you have outlined ...

• MQL5 Reference – How to use algorithmic/automated trading language for MetaTrader 5
• MQL5 programming for traders - Book on MQL5.com
• Neural Networks in Algorithmic Trading – a practical guide to using machine learning for algo trading.
  

PS! It's called—due diligence—or doing your research!

[Farhad Soltani]

How to Export and Format MQL5 Indicator Data and Trade History for Python to Train a Machine Learning Model?

Hi Hydra,

Exporting MQL5 Indicator Data to Python

Recommended approach: Instead of CSV files (which can cause sync/locking issues), use the Python MetaTrader5 library. It connects directly to the terminal.

MQL5 function: If you must export from the indicator side, use iCustom and write results with FileWrite (CSV). A more professional approach is to fetch indicator values from Python using mt5.copy_rates_from or mt5.copy_indicator.

Formatting Trade History for Training

Suggested format: Convert the data into a pandas DataFrame in Python.

Key columns:

Timestamp (primary index)

Price_Change (percentage price change)

Indicator_Value (your custom indicator values)

Target (trade outcome as binary 0/1 or a numerical value)

Note: Keep the data as a time-series (ordered chronologically), not random samples.

Machine Learning Model Recommendations

To start: use XGBoost or LightGBM — they work very well and fast on tabular trade-history data.

Time-series: if you want to predict raw price movement, consider LSTM; but to predict strategy outcome (e.g., whether a trade will be profitable), classification models like Random Forest often give more interpretable results.

Integration between MQL5 and Python

Primary library: use the official MetaTrader5 Python package.

import MetaTrader5 as mt5

# initialize connection to MetaTrader

mt5.initialize()

Real-time bridge: for higher throughput use ZeroMQ as a fast bridge between MQL5 and Python.

Best Practices in Preprocessing

Normalization: scale indicator values (different ranges) with StandardScaler or MinMaxScaler.

Feature engineering: prefer returns (price changes) or volatility metrics instead of raw prices.

Avoid overfitting: use Walk-Forward Validation — train on past data and test only on future slices (avoid random shuffling for time series).

Recommended Resources

Official docs: https://www.mql5.com/en/docs/python_metatrader5 (this is central to your work).

GitHub: search for projects named “MT5-Python-ML-Bridge”.

QuantConnect: excellent resources and examples for backtesting ML models in finance.

If you want, I can also draft a short forum post around a specific task (for example: “How to fetch custom indicator buffers into a pandas DataFrame without CSV?”) to get targeted help.

Best regards.

[Mattias Lunetti]

Farhad Soltani #:

Hi Hydra,

Exporting MQL5 Indicator Data to Python

Recommended approach: Instead of CSV files (which can cause sync/locking issues), use the Python MetaTrader5 library. It connects directly to the terminal.

MQL5 function: If you must export from the indicator side, use iCustom and write results with FileWrite (CSV). A more professional approach is to fetch indicator values from Python using mt5.copy_rates_from or mt5.copy_indicator.

Formatting Trade History for Training

Suggested format: Convert the data into a pandas DataFrame in Python.

Key columns:

Timestamp (primary index)

Price_Change (percentage price change)

Indicator_Value (your custom indicator values)

Target (trade outcome as binary 0/1 or a numerical value)

Note: Keep the data as a time-series (ordered chronologically), not random samples.

Machine Learning Model Recommendations

To start: use XGBoost or LightGBM — they work very well and fast on tabular trade-history data.

Time-series: if you want to predict raw price movement, consider LSTM; but to predict strategy outcome (e.g., whether a trade will be profitable), classification models like Random Forest often give more interpretable results.

Integration between MQL5 and Python

Primary library: use the official MetaTrader5 Python package.

Real-time bridge: for higher throughput use ZeroMQ as a fast bridge between MQL5 and Python.

Best Practices in Preprocessing

Normalization: scale indicator values (different ranges) with StandardScaler or MinMaxScaler.

Feature engineering: prefer returns (price changes) or volatility metrics instead of raw prices.

Avoid overfitting: use Walk-Forward Validation — train on past data and test only on future slices (avoid random shuffling for time series).

Recommended Resources

Official docs: https://www.mql5.com/en/docs/python_metatrader5 (this is central to your work).

GitHub: search for projects named “MT5-Python-ML-Bridge”.

QuantConnect: excellent resources and examples for backtesting ML models in finance.

If you want, I can also draft a short forum post around a specific task (for example: “How to fetch custom indicator buffers into a pandas DataFrame without CSV?”) to get targeted help.

Best regards.

Thank you, it helped me