DataForge Instrument Passport and Bar Exporter
- Utilità
-
Sayed Javad Razavi Ebrahimi
🔧 Software engineer & data analyst — professional-grade MQL5 EAs and indicators. - Versione: 2.0
DataForge exports MetaTrader 5 bars to a CSV file, and writes into that file what the data actually contains.
It does not trade. It places no orders, reads no positions, and requires no account permissions. It reads history and writes a text file.
WHAT THE EXPORT WRITES
1. An instrument passport, placed in the CSV header
Digits, point, contract size, tick value, tick size, and the minimum, step and maximum lot are written as comment lines at the top of the file. Export the same symbol from two servers and compare the two headers line by line.
This matters because those fields are not always identical across servers. A symbol can carry the same contract size and the same tick size on two servers while reporting a different tick value, which changes what one point is worth in money. Nothing in the terminal warns you when that happens.
2. A per-year volume table, read from the bars
For every year present in the history, the script reports the number of bars, how many carry a non-zero tick volume, and how many carry a non-zero real volume. This table is read with CopyRates, not from the Strategy Tester.
The two are different things. The tester's history-quality bar describes the tick stream it generates. It says nothing about the real volume stored in the bars themselves. Reading one for the other leads to building features on a field that is empty, or to discarding a field that is present.
3. Feature columns as MQL5 computes them
The export can also write a small set of computed inputs. They are not there for training: they are a reference, so that a pipeline written in another language can recompute the same inputs and compare them value by value.
WHAT THE PARITY SCRIPT DOES
The second script reads those raw inputs back out of the export file and writes them again after applying a scaler that you supply. It deliberately does not recompute them.
The reason is simple. If a check recomputes the inputs, it ends up testing the computation a second time instead of testing the normalisation, and any difference of depth or of moment between the two runs shows up as a failure with the wrong name. Reading the same lines removes that ambiguity.
The scaler file carries its own stage and frame identifiers in its header, and the script verifies them. A scaler that does not match is refused rather than used silently.
REPORTED ALSO
- The first and last bar actually read, published rather than assumed.
- The number of warm-up bars, which come out as "nan" by design.
- The count of columns, taken from the scaler rather than from a constant.
- Timestamps in ISO order, so that a parser cannot confuse day and month.
NOTES AND LIMITS
- Designed and tested on the M15 period. Other periods are untested.
- Order-book depth was empty on every CFD broker tested. MarketBookAdd is accepted and returns nothing.
- No tick carried a buy or sell flag on any broker tested, so no order-flow delta is measurable from this data. A signed volume built from the tick rule is an inference, not a measurement.
- Before exporting, set "Max bars in chart" to unlimited in the terminal options and scroll the chart back so that the history is loaded. A script reads what the terminal has cached, and the cap truncates without raising an error.
COMPANION SCRIPT, PUBLISHED SEPARATELY
A second script, ForgeParity, is published as open source alongside this one. The address is on the author profile. It reads the exported inputs back and writes them again after applying a scaler you supply, so that a normalisation drifting between two chains cannot pass unnoticed.
It is not part of this product, and deliberately so: it needs a scaler file that your own training code produces, and without one it refuses to run. A tool that does nothing on first open does not belong in a product listing. The full source of both scripts is published for reference.
It does not trade. It places no orders, reads no positions, and requires no account permissions. It reads history and writes a text file.
WHAT THE EXPORT WRITES
1. An instrument passport, placed in the CSV header
Digits, point, contract size, tick value, tick size, and the minimum, step and maximum lot are written as comment lines at the top of the file. Export the same symbol from two servers and compare the two headers line by line.
This matters because those fields are not always identical across servers. A symbol can carry the same contract size and the same tick size on two servers while reporting a different tick value, which changes what one point is worth in money. Nothing in the terminal warns you when that happens.
2. A per-year volume table, read from the bars
For every year present in the history, the script reports the number of bars, how many carry a non-zero tick volume, and how many carry a non-zero real volume. This table is read with CopyRates, not from the Strategy Tester.
The two are different things. The tester's history-quality bar describes the tick stream it generates. It says nothing about the real volume stored in the bars themselves. Reading one for the other leads to building features on a field that is empty, or to discarding a field that is present.
3. Feature columns as MQL5 computes them
The export can also write a small set of computed inputs. They are not there for training: they are a reference, so that a pipeline written in another language can recompute the same inputs and compare them value by value.
WHAT THE PARITY SCRIPT DOES
The second script reads those raw inputs back out of the export file and writes them again after applying a scaler that you supply. It deliberately does not recompute them.
The reason is simple. If a check recomputes the inputs, it ends up testing the computation a second time instead of testing the normalisation, and any difference of depth or of moment between the two runs shows up as a failure with the wrong name. Reading the same lines removes that ambiguity.
The scaler file carries its own stage and frame identifiers in its header, and the script verifies them. A scaler that does not match is refused rather than used silently.
REPORTED ALSO
- The first and last bar actually read, published rather than assumed.
- The number of warm-up bars, which come out as "nan" by design.
- The count of columns, taken from the scaler rather than from a constant.
- Timestamps in ISO order, so that a parser cannot confuse day and month.
NOTES AND LIMITS
- Designed and tested on the M15 period. Other periods are untested.
- Order-book depth was empty on every CFD broker tested. MarketBookAdd is accepted and returns nothing.
- No tick carried a buy or sell flag on any broker tested, so no order-flow delta is measurable from this data. A signed volume built from the tick rule is an inference, not a measurement.
- Before exporting, set "Max bars in chart" to unlimited in the terminal options and scroll the chart back so that the history is loaded. A script reads what the terminal has cached, and the cap truncates without raising an error.
COMPANION SCRIPT, PUBLISHED SEPARATELY
A second script, ForgeParity, is published as open source alongside this one. The address is on the author profile. It reads the exported inputs back and writes them again after applying a scaler you supply, so that a normalisation drifting between two chains cannot pass unnoticed.
It is not part of this product, and deliberately so: it needs a scaler file that your own training code produces, and without one it refuses to run. A tool that does nothing on first open does not belong in a product listing. The full source of both scripts is published for reference.
