For a sustainable EA, how long period back test should be needed at least? (with 99% high quality back test data) - page 3

New comment
 
Marco vd Heijden:

You mean sustainable in the tester ?

A backtest say's absolutely nothing about potential future performance. 

Thank you so much, Marco. 

About sustainable, of course not in the tester but for real. 

I just can not understand that, if an EA could not pass the past backtest, why it could pass the challenge in the future... 
 
Sky L:

Thank you, Anak. 

Do you mean EA should be update timely, and it have to pass one year back test? 

Just as i say before great EA would passed backtest several years before now. Future market may change but who knows it? Instead of worrying about future market price action that no one knows, we can only bet on a profitable system which everyone do in this market. If you see some signals there are some that is profitable year over year with just sticking to one system without changing it. Changing a long term profitable system just because of weak short term performance is just foolish. But if profitable system doesn't make profit anymore in one year then maybe that system is doomed
 
Sky L:

Thank you so much, Marco. 

About sustainable, of course not in the tester but for real. 

I just can not understand that, if an EA could not pass the past backtest, why it could pass the challenge in the future... 
I dont understand the logic of that too. How can you be sure that it will be successful if it failed in the past? If you dont believe backtest why create a system at all which can be proven by backtest before starting real. I'm sure profitable EA on live will have profitable performance in backtest.
 

Interesting topic . I will share my opinion (which is an opinion , its not fact ,its not tested)

Analogy : 

Imagine a corridor with several rooms .

  1. Each Room is dark and has a sculpture in the middle ,situated on a pedestal.Thats the only thing we know.
  2. We are given a sheet of tinfoil and we are tasked to go inside each room and wrap the tinfoil sheet around the sculpture.
  3. If the room is the last room we are able to take a guess about what the sculpture is while inside the room while wrapping.
  4. If the room is not the last room we are able to take a guess once we come out and look at the shape the tinfoil sheet has.
  5. We have the privilege of being allowed to turn on the lights at any point when we are in any room other than the last.

So lets say room 2010 has a sculpture of a cat.The Last room is room 2021.

We can turn the light on all rooms but room 2021.

If you then go into room 2010 ,wrap the foil sheet around the cat with the light on and then proceed to room 2011 and before turning the light on you assume the shape of the tinfoil sheet will match the sculputre inside room 2011 ,are you right ?

If you create a machine that wraps the foil around sculptures perfectly ,then once done,comes out with a perfect tinfoil shape of what was inside ,are you right ?

If you create a machine that learns to predict the shape of the sculpture while wrapping the tinfoil around it ,are you right ?

 
Lorentzos Roussos:

Interesting topic . I will share my opinion (which is an opinion , its not fact ,its not tested)

Analogy : 

Imagine a corridor with several rooms .

  1. Each Room is dark and has a sculpture in the middle ,situated on a pedestal.Thats the only thing we know.
  2. We are given a sheet of tinfoil and we are tasked to go inside each room and wrap the tinfoil sheet around the sculpture.
  3. If the room is the last room we are able to take a guess about what the sculpture is while inside the room while wrapping.
  4. If the room is not the last room we are able to take a guess once we come out and look at the shape the tinfoil sheet has.
  5. We have the privilege of being allowed to turn on the lights at any point when we are in any room other than the last.

So lets say room 2010 has a sculpture of a cat.The Last room is room 2021.

We can turn the light on all rooms but room 2021.

If you then go into room 2010 ,wrap the foil sheet around the cat with the light on and then proceed to room 2011 and before turning the light on you assume the shape of the tinfoil sheet will match the sculputre inside room 2011 ,are you right ?

If you create a machine that wraps the foil around sculptures perfectly ,then once done,comes out with a perfect tinfoil shape of what was inside ,are you right ?

If you create a machine that learns to predict the shape of the sculpture while wrapping the tinfoil around it ,are you right ?

I am delighted with this analogy for those who can read between the lines 😆
 
Lorentzos Roussos:

Interesting topic . I will share my opinion (which is an opinion , its not fact ,its not tested)

Analogy : 

Imagine a corridor with several rooms .

  1. Each Room is dark and has a sculpture in the middle ,situated on a pedestal.Thats the only thing we know.
  2. We are given a sheet of tinfoil and we are tasked to go inside each room and wrap the tinfoil sheet around the sculpture.
  3. If the room is the last room we are able to take a guess about what the sculpture is while inside the room while wrapping.
  4. If the room is not the last room we are able to take a guess once we come out and look at the shape the tinfoil sheet has.
  5. We have the privilege of being allowed to turn on the lights at any point when we are in any room other than the last.

So lets say room 2010 has a sculpture of a cat.The Last room is room 2021.

We can turn the light on all rooms but room 2021.

If you then go into room 2010 ,wrap the foil sheet around the cat with the light on and then proceed to room 2011 and before turning the light on you assume the shape of the tinfoil sheet will match the sculputre inside room 2011 ,are you right ?

If you create a machine that wraps the foil around sculptures perfectly ,then once done,comes out with a perfect tinfoil shape of what was inside ,are you right ?

If you create a machine that learns to predict the shape of the sculpture while wrapping the tinfoil around it ,are you right ?

Thank you so much, Lorentzos. 


>>If you then go into room 2010 ,wrap the foil sheet around the cat with the light on and then proceed to room 2011 and before turning the light on you assume the shape of the tinfoil sheet will match the sculputre inside room 2011 ,are you right ?

We don't need to fit everywhere of the sculpture by one tinfoil, that's almost impossible. 

More likely we have to find the similar point of almost sculptures, for example, the bottom of the sculptures always flight (to fit to the pedestal),

then what we are trying is to set our tinfoil between the sculptures and the pedestal. 


>>If you create a machine that wraps the foil around sculptures perfectly ,then once done,comes out with a perfect tinfoil shape of what was inside ,are you right ?

It depends on your approach, cover it on the sculpture, put it into the sculpture, set it under the sculpture, you have to find a way and make a rule by yourself. 

"Perfectly" is not needed, "creativeness" always needed. 


>>If you create a machine that learns to predict the shape of the sculpture while wrapping the tinfoil around it ,are you right ?

Deep learning is important, totally agree with you. 

By my experience of dozens EAs, all of EAs need to improve in time...

 

Great topic. 

To be honest. I don't have exact answer for this but I would like to comment some of my findings and get involve in this discussion. Haha

Number of Trades

5 years or 10 years. I belief about the number of trades within the time range. If we're backtesting 10 years but we only have total 10 trades within that 10 years. Is that even statistically proven? I tried to google around, some recommend 30 trades is the minimum while some recommend 100 trades. I still finding a practical method to identify number of trades required for the strategy. a math? a formula ? maybe ?

Market Condition Clustering

Recently, I came across a clustering model. I use an indicator and tried to group the market to different state (ranging, trending, volatile, market condition and etc). Instead of backtest the strategy for 10 years in EURUSD, I would backtest the strategy starting from 2016 for EURUSD. This is because the clustering model show me that the market starting from 2016 is different compare to the previous one. How accurate is this ? I'm not sure. I just trying and research everyday. Keep losing keep failing and success later? Lol.

Walk Forward Matrix

This is another term that I came across as well. In layman term, it split your history data and backtest to find out the best possible timeframe to optimize your EA again. It seems like a powerful tools but I seldom use it due to my computer processing is too low. It takes days to finish a testing for 1 strategy.

 
Guan Chuan Lee:

Great topic. 

To be honest. I don't have exact answer for this but I would like to comment some of my findings and get involve in this discussion. Haha

Number of Trades

5 years or 10 years. I belief about the number of trades within the time range. If we're backtesting 10 years but we only have total 10 trades within that 10 years. Is that even statistically proven? I tried to google around, some recommend 30 trades is the minimum while some recommend 100 trades. I still finding a practical method to identify number of trades required for the strategy. a math? a formula ? maybe ?

Market Condition Clustering

Recently, I came across a clustering model. I use an indicator and tried to group the market to different state (ranging, trending, volatile, market condition and etc). Instead of backtest the strategy for 10 years in EURUSD, I would backtest the strategy starting from 2016 for EURUSD. This is because the clustering model show me that the market starting from 2016 is different compare to the previous one. How accurate is this ? I'm not sure. I just trying and research everyday. Keep losing keep failing and success later? Lol.

Walk Forward Matrix

This is another term that I came across as well. In layman term, it split your history data and backtest to find out the best possible timeframe to optimize your EA again. It seems like a powerful tools but I seldom use it due to my computer processing is too low. It takes days to finish a testing for 1 strategy.

Thank you so much, Guan. 


Your experience is very close to my experience. I'm surprised for that. 

Could you give me a quick guidance about Walk Forward Matrix? 

123
New comment