Past tester data reliabilty

To add comments, please log in or register
Gerry
44
Gerry  

Hi,

I am wondering if testing EAs on past data using the strategy tester application is reliable? The problem I have is that I have both a live account and a demo account. I did all my testing on the live account strategy tester, but when I ran the same EA on the demo account tester I get different results.

The EA I use takes trades based on a simple algorithm and also records data (eg entry price of a trade taken, drawdown price etc) and stores these values to arrays, and then at the end sends the data to an excel file.

When I do this on the "live" account vs the "demo" account the results are different. Now, I say different but only slightly maybe 3 or 4 pips for infrequent trades but they are the same for most of the trades. However, even though the differences are slight and not frequent it is still enough to throw off the trading results considerably.

The differences are non-negligible. And also I would also want to say that variability in spread is not the cause here since 1. I run both one after the other meaning spread is almost the same 2. The values for which there is a discrepancy are market prices (bid) which shouldn't be dependent on the spread.

It is annoying, as now I have no failth in my backtested results.

Any experience with this problem? The market I did this on was for example Gbpusd and GoMarkets is the broker.

I can't see my EA being the problem since I used the same EA across both accounts.


best,

G.

Note that for some reason the demo only allows testing (on GbpUsd) as far back as the beginning of August 2012, yet my live account goes back on GbpUsd to 25 July 2011.

Ubzen
5299
Ubzen  

If you and I ran that same Expert, we'd get different results. Matter of fact, if you ran 2-Terminal with that same Expert, you could get different results. Are we to conclude that we have no faith in Live testing?

As to the ""why" would it be different" because that'll most likely be your next question. Try doing some Searching within this forum, I and others have covered this topic extensively. Also, reading some material on how the tester [any tester] works would be helpful. One example of the "why" answer is Slippage.

I'm currently involved in such discussion. Here.

Simon Gniadkowski
17239
Simon Gniadkowski  
14967057:

Hi,

I am wondering if testing EAs on past data using the strategy tester application is reliable? The problem I have is that I have both a live account and a demo account. I did all my testing on the live account strategy tester, but when I ran the same EA on the demo account tester I get different results.

The EA I use takes trades based on a simple algorithm and also records data (eg entry price of a trade taken, drawdown price etc) and stores these values to arrays, and then at the end sends the data to an excel file.

When I do this on the "live" account vs the "demo" account the results are different. Now, I say different but only slightly maybe 3 or 4 pips for infrequent trades but they are the same for most of the trades. However, even though the differences are slight and not frequent it is still enough to throw off the trading results considerably.

The differences are non-negligible. And also I would also want to say that variability in spread is not the cause here since 1. I run both one after the other meaning spread is almost the same 2. The values for which there is a discrepancy are market prices (bid) which shouldn't be dependent on the spread.

It is annoying, as now I have no failth in my backtested results.

What was the Spread on the run on the Demo account ? what was the Spread on the run you did on the Live account ?  is your History data exactly the same ? did you copy it from one account to the other ?

If you expect the same result from 2 terminals you need exactly the same conditions on both terminals . . .  same data,  same Spread, same start and end dates,  same settings for the EA ,  same EA.

You are probably correct to have no faith in your backtest results as you probably don't know what the spread was,  how good your data is and you probably have only tested on 6 months of EURUSD data.
To add comments, please log in or register