Backtesting - Different Results from Different Brokers - same time period, chart, timeframe

rvl16
12
rvl16  

Hi All,

I seem to have run into a problem that has been discussed multiple times, but never appears to be properly resolved. Majority of previous discussions have been on the MT4 platform, however I am seeing similar issues with the MT5 Platform. 

Backtesting the same EA, over the same historical timeframe and the same chart time (1 hour, 2 hour etc) produces different results from different brokers using modelling based on real ticks, every tick. And not only are EA results different, the historical data is actually different. For example on an H1 AUD/USD chart 1st September 2020 at 18:00 GMT+0, the metatrader demo account data shows:

Open: 0.73792
High: 0.73833
Low: 0.73712
Close: 0.73740

While a live account with IC-Markets shows:

Open: 0.73788
High: 0.73837
Low: 0.73718
Close: 0.73744

While these differences are relatively small (and from a randomly selected candle), they are causing significant differences in EA results. The results produced by an EA modelled "every tick based on real ticks" produces results that are an order of magnitude in difference (or larger in some cases) across the brokers.

My questions lie in whether this is expected behaviour for an EA (note that even using example included EA's from the MT5 platform produce different results, and so appears to be independent of EA code) and if this is expected behaviour, which set of broker data should be used for back/forward testing to determine the most accurate results? If this is expected behaviour is it then prudent to then check multiple brokers to determine the best results that an EA can achieve?


Previous threads can be found here:

https://www.mql5.com/en/forum/265438
https://www.mql5.com/en/forum/134911
https://www.mql5.com/en/forum/159215
https://www.mql5.com/en/forum/159037

Backtesting results in different results on different brokers
Backtesting results in different results on different brokers
  • 2018.07.13
  • www.mql5.com
Hello, I'm backtesting an EA, but I receive completely different results on two different brokers (GMT offset is equal to -7200 for both brokers...
Enrique Dangeroux
784
Enrique Dangeroux  

Forex == decentralized.  Different prices == different result. 

Only on central exchange prices will be the same on every broker, yet could still yield different results because of different commission charges, network latency and so on.

rvl16
12
rvl16  

I had not considered the decentralized nature of FOREX, thanks for the insight. 

I will run the EA's again on a centralized exchange instrument with different brokers to determine whether this is the only thing attributing to different results. 

For each of the EA's run, the modelled broker commissions and network latency were identical. Notwithstanding, network latency and commissioning charges should not materially affect the EA trade entrance and exit points (these EA's do not alter their behaviour based on spread etc). So even if end values are different, I'd expect the EA to make the same number and duration trades for the same historical window if data is identical between brokers.

Irwan Adnan
1111
Irwan Adnan  
rvl16:

I had not considered the decentralized nature of FOREX, thanks for the insight. 

I will run the EA's again on a centralized exchange instrument with different brokers to determine whether this is the only thing attributing to different results. 

For each of the EA's run, the modelled broker commissions and network latency were identical. Notwithstanding, network latency and commissioning charges should not materially affect the EA trade entrance and exit points (these EA's do not alter their behaviour based on spread etc). So even if end values are different, I'd expect the EA to make the same number and duration trades for the same historical window if data is identical between brokers.

I think each broker also have EA (a kind of) at server side that can handles orders from clients and the setting is different for each broker. 

rvl16
12
rvl16  
Irwan Adnan:

I think each broker also have EA (a kind of) at server side that can handles orders from clients and the setting is different for each broker. 

I'm not sure what you mean by this? are you talking about the way ECN brokers match trades in markets? Or something else?