Strategy Tester - MT4

 

Hello,

I have been struggling with Strategy Tester for some time and got to wondering if the problems I have been seeing are common -- or if I am doing something stupid...

My problem is that I get unrepeatable results.  I have done pretty well all of my testing on EUR-USD, typically over two or three years of data.  I have a number of parameters that I systematically vary from run-to-run, attempting to find a sweet spot that delivers broadly optimized results.  I realize there are problems with the wisdom of trying to tune this kind of process too finely, but I still think I should get consistent results.

Am I unique in this regard, or is it somehow expected to see big variances in the result from what should be an identical test?  For example, I may do a run with a Stop Loss set for say 30 pips, fiddle with 32, 34, 36 etc and then come back and try 30 again -- and get significantly different results.  Like BIG differences -- variations might be as different as an Expected Payoff of 20 pips changing to -10 pips.

I have tried the tester on several EA packages, different broker versions, a couple of different computers and reloaded/recalculated pip data -- all with no positive results.

In my experience, you always get the same result when you run the same computer with the same program against the same data.  Since I still have faith in computers, I would really appreciate some insight into what I might be doing wrong here... 

 

Thanks 

 
farkle:

Hello,

I have been struggling with Strategy Tester for some time and got to wondering if the problems I have been seeing are common -- or if I am doing something stupid...

My problem is that I get unrepeatable results.  I have done pretty well all of my testing on EUR-USD, typically over two or three years of data.  I have a number of parameters that I systematically vary from run-to-run, attempting to find a sweet spot that delivers broadly optimized results.  I realize there are problems with the wisdom of trying to tune this kind of process too finely, but I still think I should get consistent results.

Am I unique in this regard, or is it somehow expected to see big variances in the result from what should be an identical test?  For example, I may do a run with a Stop Loss set for say 30 pips, fiddle with 32, 34, 36 etc and then come back and try 30 again -- and get significantly different results.  Like BIG differences -- variations might be as different as an Expected Payoff of 20 pips changing to -10 pips.

I have tried the tester on several EA packages, different broker versions, a couple of different computers and reloaded/recalculated pip data -- all with no positive results.

In my experience, you always get the same result when you run the same computer with the same program against the same data.  Since I still have faith in computers, I would really appreciate some insight into what I might be doing wrong here... 

If you want the same results use the same history data, the same EA settings with the same EA, the same date ranges and the same Spread.  If you vary any of these your results may well be different.
 
RaptorUK:
If you want the same results use the same history data, the same EA settings with the same EA, the same date ranges and the same Spread.  If you vary any of these your results may well be different.

Thanks Raptor,

 I appreciate your help.  I thought I was doing all of those things -- but I guess Spread = Current is a killer!  I'll give it a good try -- it makes a ton of sense that a Current spread is variable...   

 
And thankfully the strategy tester in the latest version of MT4 allows you to set the spread, so you can test/see under what spreads your EA is profitable. Then you can code a MaxSpread param so the bot only trades when the criteria are met, including when the spread is within tolerance.
Reason: