Python or MQL5 — Code Migration - page 3

 
3195512 #: Understood. I appreciate the help!

Also, please note that OHLC bar data and tick data do not necessarily cover the same time range.

The available tick data, usually only covers the more recent time range. For example, the OHLC bar data may go back 10 years, but the tick data may only be only for the last one or two years.

You have to keep this in mind when you do back-tests.

If you want to test with "Every tick based on real ticks" modelling, then you must make sure that your selected test range is only for the time that spans the available real tick data.

If your test period is out of the tick data range, the Strategy Tester will use generated/simulated tick data instead. This is very important if you want to maintain the expected reliability.

 

Wonderful, that is very enlightening.

I will proceed with that verification. Based on my research, the amount of generated or simulated tick data should be reflected in the data quality metrics, which would allow me to perform the verification you suggested. Is that correct?


Best Regards,

João Valério


Fernando Carreiro #:

Also, please note that OHLC bar data and tick data do not necessarily cover the same time range.

The available tick data, usually only covers the more recent time range. For example, the OHLC bar data may go back 10 years, but the tick data may only be only for the last one or two years.

You have to keep this in mind when you do back-tests.

If you want to test with "Every tick based on real ticks" modelling, then you must make sure that your selected test range is only for the time that spans the available real tick data.

If your test period is out of the tick data range, the Strategy Tester will use generated/simulated tick data instead. This is very important if you want to maintain the expected reliability.

 

3195512 #:

Point 1: How can a trading strategy avoid dependency on book information / DOM and still provide realiable results?

I believe, in Nasdaq Futures, the maximum depth available is limited to bid, ask, last price, and volume per tick. DOM (Depth of Market) or book information provides a snapshot of the supply and demand for liquid, tradable assets (such as contracts in futures markets). This data is crucial for effective backtesting. Without it, you cannot accurately simulate supply and demand dynamics throughout different periods of the trading day, which would result in unrealistic outcomes.

For example, how could a strategy tester know that I would be able to sell a specific contract at a price of 20,000 without knowing the available volume? It simply cannot be done accurately without such information.

Given this, I need to verify whether the strategy tester is indeed using DOM/book data during backtesting. Do you know someone who could help on this matter or should I wait for a possible answer?

Most known strategies don't use the Order Book information or depend on Market Depth.

If you don't use any of the following functionality, then there is nothing to worry about ...

If however, you do need the above functionality, then I lack the knowledge to be able to help you. And in that case, maybe @Alain Verleyen may be able to offer you some advice, as he has more experience with Futures and AMP.

Most probably, the Strategy Tester does simulate the DOM to a degree, but I have no ideia how well. I am sure @Alain Verleyen will see this post and offer his opinion.

 

I appreciate the clarification. Hopefully  @Alain Verleyen will be able to help me as well.


I understand your point. However, what I'm stating is this: For a strategy to be backtested accurately, it must rely on Order Book information (DOM). While it's possible to develop a strategy that doesn't take DOM data into account, the backtest will never be as realistic as it would be with the inclusion of Order Book information. The strategy that considers DOM better reflects real-world conditions, as it captures the true dynamics of supply and demand.


Best Regards,

João Valério


Fernando Carreiro #:

Most known strategies don't use the Order Book information or depend on Market Depth.

If you don't use any of the following functionality, then there is nothing to worry about ...

If however, you do need the above functionality, then I lack the knowledge to be able to help you. And in that case, maybe @Alain Verleyen may be able to offer you some advice, as he has more experience with Futures and AMP.

Most probably, the Strategy Tester does simulate the DOM to a degree, but I have no ideia how well. I am sure @Alain Verleyen will see this post and offer his opinion.


Alain Verleyen
Alain Verleyen
  • 2024.09.28
  • www.mql5.com
Trader's profile
 
3195512 #:

I appreciate the clarification. Hopefully  @Alain Verleyen will be able to help me as well.


I understand your point. However, what I'm stating is this: For a strategy to be backtested accurately, it must rely on Order Book information (DOM). While it's possible to develop a strategy that doesn't take DOM data into account, the backtest will never be as realistic as it would be with the inclusion of Order Book information. The strategy that considers DOM better reflects real-world conditions, as it captures the true dynamics of supply and demand.


Best Regards,

João Valério


MT5 doesn't record historical DOM data, only ticks data (and OHLC obviously).

So to be able to backtest a strategy using DOM data, you will need to provide the DOM data yourself, this can be done using a custom symbol. You need either to record the DOM data or buy DOM data, then provides these data to a custom symbol, then you will be able to backtest your strategy with DOM data.

See the Custom Symbol API : https://www.mql5.com/en/docs/customsymbols/custombookadd

Documentation on MQL5: Custom Symbols / CustomBookAdd
Documentation on MQL5: Custom Symbols / CustomBookAdd
  • www.mql5.com
Passes the status of the Depth of Market for a custom symbol. The function allows broadcasting the Depth of Market as if the prices arrive from a...
 
3195512 #: Wonderful, that is very enlightening. I will proceed with that verification. Based on my research, the amount of generated or simulated tick data should be reflected in the data quality metrics, which would allow me to perform the verification you suggested. Is that correct?

No need for such indirect method to find the real tick data time range. What I usually do is open up the Symbol's dialogue box (Ctrl-U), and request a view of a small initial time range of tick data in which I am interested in testing.

Sometimes, it may take a while to download if it is the first time you are doing it, and you will have to be patient. But once it is complete, scroll to the bottom to see which is the earliest available tick for the time period you selected in the request.

If they match, then you are in luck and the available range is available. If not, then you will have to adjust your test period to only the available time range.

The display is limited in the number of ticks you can view, so choose a small time range to view. In the following example, I chose a single hour's worth of data from 2nd January 2024, for the E-mini NASDAQ-100.


 

Got it. Thank you very much for the explanation!


Fernando Carreiro #:

No need for such indirect method to find the real tick data time range. What I usually do is open up the Symbol's dialogue box (Ctrl-U), and request a view of a small initial time range of tick data in which I am interested in testing.

Sometimes, it may take a while to download if it is the first time you are doing it, and you will have to be patient. But once it is complete, scroll to the bottom to see which is the earliest available tick for the time period you selected in the request.

If they match, then you are in luck and the available range is available. If not, then you will have to adjust your test period to only the available time range.

The display is limited in the number of ticks you can view, so choose a small time range to view. In the following example, I chose a single hour's worth of data from 2nd January 2024, for the E-mini NASDAQ-100.

 

Thank you for your help! In the meantime, I found the same information in the documentation.

I had connected to my AMP account and received the DOM data for free (see 1.png and 2.png) for the @ENQ symbol.

In this case, should I separate the tick data from the DOM information, or can I continue using the same symbol (@ENQ) as before and simply update my code to account for the order book data as well?

Alain Verleyen #:

MT5 doesn't record historical DOM data, only ticks data (and OHLC obviously).

So to be able to backtest a strategy using DOM data, you will need to provide the DOM data yourself, this can be done using a custom symbol. You need either to record the DOM data or buy DOM data, then provides these data to a custom symbol, then you will be able to backtest your strategy with DOM data.

See the Custom Symbol API : https://www.mql5.com/en/docs/customsymbols/custombookadd

Files:
1.png  39 kb
2.png  44 kb
 
3195512 #: Thank you for your help! In the meantime, I found the same information in the documentation. I connected to my AMP account and it seems I received the DOM data for free (see 1.png and 2.png) for the @ENQ symbol. In this case, should I separate the tick data from the DOM information, or can I continue using the same symbol (@ENQ) as before and simply update my code to account for the order book data as well?
I think we may be having a problem with definitions. I consider that the tick data, not the DOM data. As @Alain Verleyen already explain, historical DOM data is not available via the standard methods provided by MetaTrader.
 

But isn't the DOM historical data the bid, ask, last, volume per each tick? This is the information shown in the Depth of Market in live trading.


Or am I misunderstanding your point?



Fernando Carreiro #:
I think we may be having a problem with definitions. I consider that the tick data, not the DOM data. As @Alain Verleyen already explain, historical DOM data is not available.