Download MetaTrader 5

Bug in the Strategy Tester (MT4, build 950)

To add comments, please log in or register
Unable to find the answer to your question? Use the search!
Paolo Miocchi
18773
Paolo Miocchi 2016.02.10 15:33 

Hello,

I'm using Markets.com Metatrader 4.00, build 950 and I'd like to report what I think is a dangerous bug in the Strategy Tester tool. 

The Strategy Tester always opens a pending stop order at the price given by the trader when he/she placed the order, while in reality it should be opened (i.e. transformed into a market order) at the first ask/bid broker price that activated it (i.e., the first ask/bid price that goes above/below the pending order price).

Just as an example: suppose you put a buystop order on EURUSD at 1.13000, when the current ask price is 1.12950. Now suppose that, due to a gap, suddenly the ask price jumps to 1.13100.

In reality, any broker would open your order at that price, i.e. at 1.13100, while the Strategy Tester would open it at 1.13000 and consequently, according to the Tester, this order would have a profit of 100 points right away!!

This is a very dangerous bug, because in case of price gaps it could give unreal, fictitious, huge profits in backtesting!

It does not depend on the modelling quality. 

It would be very easy for MT4 programmers to fix this bug! 

Francis Dogbe
9501
Francis Dogbe 2016.02.10 20:56  
hmmm... smh
Stuart Browne
5816
Stuart Browne 2016.02.11 08:31  
Paolo Miocchi:

Hello,

I'm using Markets.com Metatrader 4.00, build 950 and I'd like to report what I think is a dangerous bug in the Strategy Tester tool. 

The Strategy Tester always opens a pending stop order at the price given by the trader when he/she placed the order, while in reality it should be opened (i.e. transformed into a market order) at the first ask/bid broker price that activated it (i.e., the first ask/bid price that goes above/below the pending order price).

Just as an example: suppose you put a buystop order on EURUSD at 1.13000, when the current ask price is 1.12950. Now suppose that, due to a gap, suddenly the ask price jumps to 1.13100.

In reality, any broker would open your order at that price, i.e. at 1.13100, while the Strategy Tester would open it at 1.13000 and consequently, according to the Tester, this order would have a profit of 100 points right away!!

This is a very dangerous bug, because in case of price gaps it could give unreal, fictitious, huge profits in backtesting!

It does not depend on the modelling quality. 

It would be very easy for MT4 programmers to fix this bug! 

I already reported a very similar problem with the T/P to service desk. The bug was closed as "it's not a bug, it's a feature"! Even though it's impossible for it to happen in real trading, the explanation was that it is a "worst case scenario" feature.

You can try reporting it to the service desk but I'm sure you'll get a similar response

Paolo Miocchi
18773
Paolo Miocchi 2016.02.11 10:51  
Stuart Browne:

I already reported a very similar problem with the T/P to service desk. The bug was closed as "it's not a bug, it's a feature"! Even though it's impossible for it to happen in real trading, the explanation was that it is a "worst case scenario" feature.

You can try reporting it to the service desk but I'm sure you'll get a similar response

Thanks Stuart,

yes, I reported to the service desk too. No reply yet.

It's a very bad bug. I think that many EAs that use pending orders and show "beautiful" Strategy Tester (ST) results are seriously affected by this bug, in the sense that they are only apparently profitable and this surely is inducing many users to lose their money!

Even if you have small price gaps on M1 that are "invisible" if you look at M30 or H1 timeframes, within the ST they could turn losing pending orders into winning ones, that summed up together give you a fictitiously high profit factor...

We really should promote a massive "message campaign" to the service desk to put pressure on them to correct this bug. That would be very easy to be fixed!!! 

Stuart Browne
5816
Stuart Browne 2016.02.11 11:53  
Totally agree Paolo. 
Paolo Miocchi
18773
Paolo Miocchi 2016.02.11 12:37  

 

 

They answered me basically stating that ST can just simulate strictly instant execution with no slippage

But I'm sure that that kind of slippage can be very easily included by the MT4 programmers!  

I would like to contact directly some MT4 Strategy Tester developer. Anybody knows how can I do? 

Alain Verleyen
Moderator
30743
Alain Verleyen 2016.02.11 17:08  

You are losing your time, Metaquotes will not change a software which is like this from the start, more than 10 years ago.

And they are right in my opinion, while I agree with you it will be more accurate, in the end it's useless to have such accuracy as anyway the Strategy Tester will never give you the same results on a real account : no slippage, no latency, fixed spread.

Paolo Miocchi
18773
Paolo Miocchi 2016.02.11 21:38  
Alain Verleyen:

You are losing your time, Metaquotes will not change a software which is like this from the start, more than 10 years ago.

And they are right in my opinion, while I agree with you it will be more accurate, in the end it's useless to have such accuracy as anyway the Strategy Tester will never give you the same results on a real account : no slippage, no latency, fixed spread.

You're right but I really don't understand why they don't do such a simple and straightforward correction! It's one million times simpler than the recent improvements they did on the latest build.

I suspect that a similar bug also affects the stoploss closing of a normal market order. Thus, I think that the best an EA developer should do to reduce the discrepancy of backtesting with the real world is to replace pending orders with market orders and normal stoploss with virtual stoploss... 

Marco vd Heijden
Moderator
4704
Marco vd Heijden 2016.02.11 22:27  
Paolo Miocchi:

You're right but I really don't understand why they don't do such a simple and straightforward correction!

I do.

And you don't seem to understand the underlying principle of what happens after someone discovers a seemingly profitable strategy as well.

Paolo Miocchi
18773
Paolo Miocchi 2016.02.11 22:45  
Marco vd Heijden:

I do.

And you don't seem to understand the underlying principle of what happens after someone discovers a seemingly profitable strategy as well.

:-(

It is also very suspicious that they didn't (straightforwardly) correct the real spread issue, given that fxt files already have in principle all the needed data...  

Marco vd Heijden
Moderator
4704
Marco vd Heijden 2016.02.12 14:24  
Paolo Miocchi:

:-(

It is also very suspicious that they didn't (straightforwardly) correct the real spread issue, given that fxt files already have in principle all the needed data...  

As far as i can see there is no need to correct anything and i see no suspicion or errors anywhere.

Maybe it's just your expectation that is way off.

To add comments, please log in or register