MT4 Backtesting with different Brokers don't match!

 

I am new in using MT4 and I am currently backtesting some EAs. I have found something very interesting in which some of you might want to verify with your own EAs.

I ran the same EA with the same presets on MT4 from metaquotes and MT4 from InterbankFX. Simulation dates etc all the same.

I found that they both don't give the same results, the graphs and times of buys and sell don't match (hence different earnings and loss). What gives! Same EAs Same presets, same pairs but different results.

Maybe some of you more experienced users might want to suggest an answer.

 

sts98ik,

what were your two brokers?

i understand Aaragorn was having similar troubles, only he got different results from the same broker, interbankfx.

different results from the demo and live.

 
Eaglehawk:
sts98ik,

what were your two brokers?

i understand Aaragorn was having similar troubles, only he got different results from the same broker, interbankfx.

different results from the demo and live.

I believe the default metaquote MT4 was using a Mig-Demo account and interbankfx a live mini account. Shouldn't we all be simulating a 'common' database where all the prices , ticks and volumes are the same?

I think I might be having the same problem as Aaragorn, same broker same account (naturally same EA, presets, same simulation period/dates) with interbankfx i.e. one day I get one result and another a different outcome . Upon closer inspection, of the visual graphs I did notice that some buy and sell positions were placed on phantom prices sometimes 2-3 pips above/below the current bar (especially if you switch the time period say from M15 to M5). The default symbol i.e. the simulated time period looks fine but the M5 buys and sells may be placed on a phantom prices (within the timeframe of the M15 bar). The backtesting was tested on every tick as oppose to using the 12-point control mode.

I am beginning to wonder if this 'bug' is caused by a price synchronsing problem over different periods or may be an interpolation issue.

Whatever the 'bug' this issue is giving a confidence issue on my EA, as certain times the simulation gives winning results vs other days resulting in losing backtested results. You would assume that running the same EA on the same data period setting etc would give the same outcome.

Warning, please check your results on the visual charts. I hope more people can feedback over this phantom price issue.

Regards

Files:
 
sts98ik:
I believe the default metaquote MT4 was using a Mig-Demo account and interbankfx a live mini account. Shouldn't we all be simulating a 'common' database where all the prices , ticks and volumes are the same?

I think I might be having the same problem as Aaragorn, same broker same account (naturally same EA, presets, same simulation period/dates) with interbankfx i.e. one day I get one result and another a different outcome . Upon closer inspection, of the visual graphs I did notice that some buy and sell positions were placed on phantom prices sometimes 2-3 pips above/below the current bar (especially if you switch the time period say from M15 to M5). The default symbol i.e. the simulated time period looks fine but the M5 buys and sells may be placed on a phantom prices (within the timeframe of the M15 bar). The backtesting was tested on every tick as oppose to using the 12-point control mode.

I am beginning to wonder if this 'bug' is caused by a price synchronsing problem over different periods or may be an interpolation issue.

Whatever the 'bug' this issue is giving a confidence issue on my EA, as certain times the simulation gives winning results vs other days resulting in losing backtested results. You would assume that running the same EA on the same data period setting etc would give the same outcome.

Warning, please check your results on the visual charts. I hope more people can feedback over this phantom price issue.

Regards

Hi,

Go to this thred of Alpari co UK. Might help answer your problems.

http://forum.alpari.co.uk/thread9.html

Wonder this is also true with MT4 user brokers in US. Invite your comments.

 

I've noticed the same thing; in my case it's IBFX data vs. Alpari demo data. Several of my EAs, especially ones that scalp, do incredibly well with Alpari data but are marginally profitable/slightly unprofitable with IBFX data. Same settings, timeframes, etc. Could this be because Alpari uses a deal desk and IBFX is closer to true interbank pricing? Any insights would be welcome...

 
brue:
I've noticed the same thing; in my case it's IBFX data vs. Alpari demo data. Several of my EAs, especially ones that scalp, do incredibly well with Alpari data but are marginally profitable/slightly unprofitable with IBFX data. Same settings, timeframes, etc. Could this be because Alpari uses a deal desk and IBFX is closer to true interbank pricing? Any insights would be welcome...

Russian Alpari and IBFX are having very different data.

Russian Alpari data are similar with Fibo Group and North Finance data. Data for IBFX is totally different.

This problem was slightly discussed on brokers' thread.

 

Different time zone

sts98ik:
I am new in using MT4 and I am currently backtesting some EAs. I have found something very interesting in which some of you might want to verify with your own EAs.

I ran the same EA with the same presets on MT4 from metaquotes and MT4 from InterbankFX. Simulation dates etc all the same.

I found that they both don't give the same results, the graphs and times of buys and sell don't match (hence different earnings and loss). What gives! Same EAs Same presets, same pairs but different results.

Maybe some of you more experienced users might want to suggest an answer.

I've also the same problem. I figured out that that's due to different time zone. MT4 is in CET (Central European Time, which is GMT+1), InterbankFX is in GMT, so there's 1 hour difference. And this one hour difference can make huge impact on the EAs if the EA trades on specific times, like my EA in Interbankfx, it trades from 8am (GMT8) and close all trades at 4pm (GMT16). If I use Alpari platform, I need to change it to trade from 9am (CET9 or GMT8) to 5pm (CET17 or GMT16).

I'm having this problem of switching times 'coz FXDD use another timeframe...different brokers in different time zones...

I hope this answers your question.

I'm looking for a script that checks the GMT not the server time so I dun need to change timeframes of trading using different brokers. If you got one, let me know. boostrade@gmail.com Thanks!

 

I developed an EA which gave great results backtesting on Alpari data for the year 2006. I then put it live on my Velocity4x account and it started losing money. I ran backtests in visual mode for the same time period on both Alpari data and Velocity4x data, and my EA opened the same losing trades on Velocity4x data, but it did not open those losing positions using Alpari data. I found that by tweaking the parameters in my EA, I was able to get good results backtesting on Velocity4x data.

Comparing charts of the two sets of data on the same time frame, there are obvious differences in the sizes and shapes of candles. I came to the conclusion that one set of data is not necessarily better for trading than the other, but the differences in price behavior require the EA to be fine-tuned to the behavior on which it will be operating.

Now there is a problem. I can download years of Alpari one-minute data, but one-minute data from Velocity4x goes back only about six weeks, not enough data to give me a high level of confidence in the back testing. Why do brokers restrict the amount of historical data that MT4 users can download? The only answer that comes readily to mind is that they do not want users to optimize their EAs such that they become "too" profitable. I e-mailed Velocity4x customer support and asked them for access to more historical data, but they did not bother to reply. So, they should not be surprised when I yank my account and start trading with Alpari when they open a branch in New York later this year.

 
brue:
I've noticed the same thing; in my case it's IBFX data vs. Alpari demo data. Several of my EAs, especially ones that scalp, do incredibly well with Alpari data but are marginally profitable/slightly unprofitable with IBFX data. Same settings, timeframes, etc. Could this be because Alpari uses a deal desk and IBFX is closer to true interbank pricing? Any insights would be welcome...

IBFX data is not as volatile, hence scalping is less effective.

I would rather use IBFX data as they are my broker, but the history download is such a clusterf**k I woulden't touch it with a barge poll.

Reason: