Can backtest results be too good to be true?

CLennon
158
Tested on nasdaq from 12th oct to 5th nov using real tick data from my own broker and low risk trades 0.5% per position

profit factor and win rate lol
Files:
William Roeder
26790
William Roeder  
Three weeks is nothing. Three years is still not enough.
Amir Yacoby
1421
Amir Yacoby  
Is it scalping?
CLennon
158
CLennon  
Amir Yacoby #:
Is it scalping?

no its a super aggressive grid system with large SL/TP 

CLennon
158
CLennon  
William Roeder #:
Three weeks is nothing. Three years is still not enough.

tested for 3 months now over 1500 trades. Surely thats enough to work with. Nasdaq price action from 3 years ago is pretty irrelevant compared to how the market moves now in comparison

William Brandon Autry
438
CLennon #:

tested for 3 months now over 1500 trades. Surely thats enough to work with. Nasdaq price action from 3 years ago is pretty irrelevant compared to how the market moves now in comparison

I think 10 years of backtesting data with no time constraints is the minimum before an EA or strategy starts to look robust enough for live use. I test mine with 15+ years of data. 3 months or 3 years isn’t near enough in my opinion. 

Tester also doesn’t account for rollover periods where spreads widen which will have a large effect on the strategy in live use. 
Cornelius Alexius Zuend
998
William Brandon Autry #:
I think 10 years of backtesting data with no time constraints is the minimum before an EA or strategy starts to look robust enough for live use. I test mine with 15+ years of data. 3 months or 3 years isn’t near enough in my opinion. 

Tester also doesn’t account for rollover periods where spreads widen which will have a large effect on the strategy in live use. 
Agreed, 10 years minimum and more if possible.

Thinking that the market dynamics from 3 years ago is irrelevant is foolish at best IMHO. 
William Brandon Autry
438
Cornelius Alexius Zuend #:
Agreed, 10 years minimum and more if possible.

Thinking that the market dynamics from 3 years ago is irrelevant is foolish at best IMHO. 
It really is. From a human prospective our minds think 3 years should be more than enough. 

Tell me this, are you the exact same person you where 3 years ago? ;) Are you the same or different with money? 
Alexandre Borela
13416
William Brandon Autry #:
I think 10 years of backtesting data with no time constraints is the minimum before an EA or strategy starts to look robust enough for live use. I test mine with 15+ years of data. 3 months or 3 years isn’t near enough in my opinion. 

Tester also doesn’t account for rollover periods where spreads widen which will have a large effect on the strategy in live use. 
Cornelius Alexius Zuend #:
Agreed, 10 years minimum and more if possible.

Thinking that the market dynamics from 3 years ago is irrelevant is foolish at best IMHO. 
Disagree, + years of backtesting will not make the EA more reliable, checking these will:

1. Was is tested only in a favorable scenario? In his case, he's using a grid system that only send buy orders, so I would expect it to blow in a bear market.
If you pick a symbol that has been in a uptrend for a long time(could be years) with little retraction even during crashes, it's obvious that buy and hold will work.

2. Was it curve fitted? Lots of people are curve fitting their EA and saying that they are optimizing it.

3. How many trades did it make? The less trades it makes, the bigger the chance for random results and curve fitting.
William Brandon Autry
438
Alexandre Borela #:
Disagree, + years of backtesting will not make the EA more reliable, checking these will:

1. Was is tested only in a favorable scenario? In his case, he's using a grid system that only send buy orders, so I would expect it to blow in a bear market.
If you pick a symbol that has been in a uptrend for a long time(could be years) with little retraction even during crashes, it's obvious that buy and hold will work.

2. Was it curve fitted? Lots of people are curve fitting their EA and saying that they are optimizing it.

3. How many trades did it make? The less trades it makes, the bigger the chance for random results and curve fitting.
As I posted above. 10+ years minimum with no time constraints will show if you even have something to consider running live. I prefer 15+ years. Testing on a demo is the next step. 

This length of time will reveal MANY possible scenarios. Testing on many pairs is also necessary. But tester alone isn’t very accurate. Demo accounts (instant execution) are better but still aren’t as accurate as a live account where slippage exists. 

If the user is adjusting or manipulating to show good results, instead of perfecting the strategy, this will soon show in real world use. It won’t take long to deplete a live account. 
Alexandre Borela
13416
William Brandon Autry #:
As I posted above. 10+ years minimum with no time constraints will show if you even have something to consider running live. I prefer 15+ years. Testing on a demo is the next step.

66 years of the S&P500, his EA would probably look great.

This is why I said that saying "running for X years" will not make sure the EA is consistent, I would rather focus on testing
the EA on unfavorable scenarios to see if it can detect them to stay away and calculate how many bad trades it would
take to blow up the account.