Historical spread during backtesting in the strategy tester is different from the historical spread stored in the Mql rates array. What are they? Why they are different? - page 2

 
FinanceEngineer:
Thanks for suggestion. For the discrepancy from the daily shift will be definitely combated when we remove the 3 or 4 minutes lags of bar forming time.

If you want to compare spread, you have to be sure you compare the right values, which is obviously not the case. If you code for multicurrency you have to ensure synchronisation of data. Some symbol can have missing bar on some timeframe.

The spread on the Strategy Tester comes from M1 historical data. The spread registered with M1 data, is the higher value of the spread during the minute.

I create a little EA to check that, which use 4 methods to find the spread. I test it on EURUSD and NZDUSD and don't found any discrepancy.

 
angevoyageur:

If you want to compare spread, you have to be sure you compare the right values, which is obviously not the case. If you code for multicurrency you have to ensure synchronisation of data. Some symbol can have missing bar on some timeframe.

The spread on the Strategy Tester comes from M1 historical data. The spread registered with M1 data, is the higher value of the spread during the minute.

I create a little EA to check that, which use 4 methods to find the spread. I test it on EURUSD and NZDUSD and don't found any discrepancy.

This is good example. I didn't know the presence of the SymbolInfoTick methods. Thanks. 

The unique problem to get the historical spreads for the multiple currency was that waiting for all 12 pairs to form the new H1 bar created some lagging like 3 to 4 minutes. Therefore the obtained historical spreads from ask - bid was actaully the spread after 3 or 4 minutes later.

They differ from the historical spreads obtained from the Mql rates methdos as the stored spread in the Mql rates methods is the spread at the first minutes of the bar.  As figurelli and I pointed out, the weired thing was the discrepancy was happend only to the bar just befor the day shift. So every 24th bar has the discrepancy. You can check this from the Excel file I uploaded in the earlier post.

In theory, the 3 or 4 mintues lagging at the beginning of new bar forming should be distributed randomly accross a day. This is still mysterious to me.  :)                    

 
angevoyageur:

If you want to compare spread, you have to be sure you compare the right values, which is obviously not the case. If you code for multicurrency you have to ensure synchronisation of data. Some symbol can have missing bar on some timeframe.

The spread on the Strategy Tester comes from M1 historical data. The spread registered with M1 data, is the higher value of the spread during the minute.

I create a little EA to check that, which use 4 methods to find the spread. I test it on EURUSD and NZDUSD and don't found any discrepancy.

Very good, this EA is quite simple and efficient.
 
FinanceEngineer:

This is good example. I didn't know the presence of the SymbolInfoTick methods. Thanks. 

The unique problem to get the historical spreads for the multiple currency was that waiting for all 12 pairs to form the new H1 bar created some lagging like 3 to 4 minutes. Therefore the obtained historical spreads from ask - bid was actaully the spread after 3 or 4 minutes later.

They differ from the historical spreads obtained from the Mql rates methdos as the stored spread in the Mql rates methods is the spread at the first minutes of the bar.  As figurelli and I pointed out, the weired thing was the discrepancy was happend only to the bar just befor the day shift. So every 24th bar has the discrepancy. You can check this from the Excel file I uploaded in the earlier post.

In theory, the 3 or 4 mintues lagging at the beginning of new bar forming should be distributed randomly accross a day. This is still mysterious to me.  :)                    

Do you run my code ? How many discrepancy do you have for each symbol ?
 
angevoyageur:
Do you run my code ? How many discrepancy do you have for each symbol ?

Yes, of course I did. I tested with 3000 bars and I don't see any discrepancy from your code. The discrepancy from my code was due to the waiting time of 3 to 6 minutes for all 12 pairs to form new H1 bar. Certainly some pairs they reform new bar later than other pairs as some pairs are less active than the other pairs. So for all 12 pairs to complete reform new bar, there are naturally some delays. In results of this the quoted spread was rather the spread of 3 or 4 minutes later since the bar opening time. So I can confirm there is no discrepancy between different method of getting spread as long as the quotation is done at the same time.

Since I removed the waiting time from my code, I don't get any more discrepancy.  Big thanks.

Documentation on MQL5: Standard Constants, Enumerations and Structures / Trade Constants / Position Properties
Documentation on MQL5: Standard Constants, Enumerations and Structures / Trade Constants / Position Properties
  • www.mql5.com
Standard Constants, Enumerations and Structures / Trade Constants / Position Properties - Documentation on MQL5
 
FinanceEngineer:

Yes, of course I did. I tested with 3000 bars and I don't see any discrepancy from your code. The discrepancy from my code was due to the waiting time of 3 to 6 minutes for all 12 pairs to form new H1 bar. Certainly some pairs they reform new bar later than other pairs as some pairs are less active than the other pairs. So for all 12 pairs to complete reform new bar, there are naturally some delays. In results of this the quoted spread was rather the spread of 3 or 4 minutes later since the bar opening time. So I can confirm there is no discrepancy between different method of getting spread as long as the quotation is done at the same time.

Since I removed the waiting time from my code, I don't get any more discrepancy.  Big thanks.

You are welcome.

A last remark, your original code where you are using MqlRates for each symbol have also some problem. Your are using the H1 open time with CopyMqlRates on M1 timeframe. The open time for H1 bar is always something like 01:00, 02:00, 16:00, 17:00, etc...But sometimes there is no M1 bar at this time. For example, the first M1 bar in the hour beginning at 16:00, could be 16:01 or 16:02 if 1 or 2 bar are missing. The spread returned by MqlRates corresponds to which time in this case ? From documentation :

When requesting data by the start date and the number of required elements, only data whose date is less (earlier) or equal to the date specified. It means, the open time of any bar, for which value is returned (volume, spread, value on the indicator buffer, prices Open, High, Low, Close or open time Time) is always less or equal to the specified one.

This problem also leads to some issue in your comparison of the spread.
 
angevoyageur:

You are welcome.

A last remark, your original code where you are using MqlRates for each symbol have also some problem. Your are using the H1 open time with CopyMqlRates on M1 timeframe. The open time for H1 bar is always something like 01:00, 02:00, 16:00, 17:00, etc...But sometimes there is no M1 bar at this time. For example, the first M1 bar in the hour beginning at 16:00, could be 16:01 or 16:02 if 1 or 2 bar are missing. The spread returned by MqlRates corresponds to which time in this case ? From documentation :

This problem also leads to some issue in your comparison of the spread.
That is great insight. Thanks. Things are even more clear.
 

Hi angevoyageur, When i run the code it does report discrepancy found.

 2014.06.27 00:17:06.481 CheckStrategyTesterSpread (USDJPY,M1) Count discrepancy on spread for USDJPY = 1 (on 2 values)

2014.06.27 00:17:06.481 CheckStrategyTesterSpread (USDJPY,M1) CHECK : time = 2014.06.27 02:16:39 Spread 1 (SymbolInfoDouble) = 8 Spread 2 (SymbolInfoTick) = 8 Spread 3 (MqlRates) = 11 Spread 4 (CopySpread) = 11

I am using Alpari Demo Server and also tried the MetaQuote-Demo Server.. same result.

How can i get the actual opening spread of each historical 1 Minute bar? 

 
delson:

Hi angevoyageur, When i run the code it does report discrepancy found.

 2014.06.27 00:17:06.481 CheckStrategyTesterSpread (USDJPY,M1) Count discrepancy on spread for USDJPY = 1 (on 2 values)

2014.06.27 00:17:06.481 CheckStrategyTesterSpread (USDJPY,M1) CHECK : time = 2014.06.27 02:16:39 Spread 1 (SymbolInfoDouble) = 8 Spread 2 (SymbolInfoTick) = 8 Spread 3 (MqlRates) = 11 Spread 4 (CopySpread) = 11

I am using Alpari Demo Server and also tried the MetaQuote-Demo Server.. same result.

How can i get the actual opening spread of each historical 1 Minute bar? 

This code is to be run on the Strategy Tester.
 

Sorry for the blast from the past, but I fear that either something has changed in mql5, or I have missed some piece.


But running Alain's EA several years later, it looks like CopyRates and CopySpread are no longer working as expected. Please look at the attached backtesting log file.



Files:
20240916.log  215 kb