How reliable is MT4 backtesting after using paid tick data?

[[Deleted]]

I have been trading forex manually for over a year now, and I consider myself intermediate. Two months back, I paired up with my friend, an advanced coder and doesn't know a lot about forex strategies; together with my scalping strategy and coding skills, we programmed an EA. Over the course, we kept optimizing it to the tiniest details and tried our best to replicate my trading strategies 1:1. Yesterday he claimed that we had reached the final build and couldn't optimize it anymore for now. I went through the code one last time, and it kinda made sense to me since I could figure out 'some' stuff out of the code. Now when I backtested it, I had very mixed feelings. It performed very well. Too well. I was equally stunned and expected something that after the efforts we both had put into it. I don't know a lot about EAs or coding, but I know that the stock MT4 backtests are often not reliable and are misleading, so I went ahead and bought a set of tick data from a paid service- Tick Data Suite 2. Now when I was sure that the EA would not perform very well, it surprisingly did! Although not as well as it did on the stock backtest, it was impressive. To confirm, I backtested it with tick data from several different sources, and the results were consistent. Although I'll forward test it now, I still want to hear the community's thoughts about this and if the paid tick data is reliable. Since I am new at automated trading, the results look too good to be true. I have attached the report to this. Please let me know if I should know anything about EAs I might not know already and if they are profitable in the long run.

[Sergey Golubev]

Forum on trading, automated trading systems and testing trading strategies

What is the best way to do an MT4 back test?

Sergey Golubev, 2020.09.19 13:28

The best way to do MT4 backtest is convert your EA to MT5 and provide MT5 backtest with "every tick based on the real tick" mode.

----------------

Testing trading strategies on real ticks and the explanation is on this post. 

[Carl Schreiber]

ItzCayden:

I have been trading forex manually for over a year now, and I consider myself intermediate. Two months back, I paired up with my friend, an advanced coder and doesn't know a lot about forex strategies; together with my scalping strategy and coding skills, we programmed an EA. Over the course, we kept optimizing it to the tiniest details and tried our best to replicate my trading strategies 1:1. Yesterday he claimed that we had reached the final build and couldn't optimize it anymore for now. I went through the code one last time, and it kinda made sense to me since I could figure out 'some' stuff out of the code. Now when I backtested it, I had very mixed feelings. It performed very well. Too well. I was equally stunned and expected something that after the efforts we both had put into it. I don't know a lot about EAs or coding, but I know that the stock MT4 backtests are often not reliable and are misleading, so I went ahead and bought a set of tick data from a paid service- Tick Data Suite 2. Now when I was sure that the EA would not perform very well, it surprisingly did! Although not as well as it did on the stock backtest, it was impressive. To confirm, I backtested it with tick data from several different sources, and the results were consistent. Although I'll forward test it now, I still want to hear the community's thoughts about this and if the paid tick data is reliable. Since I am new at automated trading, the results look too good to be true. I have attached the report to this. Please let me know if I should know anything about EAs I might not know already and if they are profitable in the long run.

Just start your EA running on a demo account with one(or more) broker (to compare). THen start real account with the smallest passible lot size (0.01) and see ...

[Eleni Anna Branou]

Your backtest period is less than 1 month, don't get too excited over such short term results.

Repeat your backtest in a larger period of some months or a year and check the results.

[Koh Kok Yeow]

I'm not sure, but many people seem to value it.

I myself don't really believe in it.  Past performance is not indicative of future performance.  Mapping current performance onto past conditions is hardly something I value as well given that market conditions are always changing.

Just my two cents.  

[[Deleted]]

Eleni Anna Branou:

Your backtest period is less than 1 month, don't get too excited over such short term results.

Repeat your backtest in a larger period of some months or a year and check the results.

I did backtest it for a year, and the results seemed incredibly unrealistic. The win ratio stayed over 90%, but the returns were overwhelming. It's my first time using an EA, so I am still a little skeptical about the returns this EA has. I still can't figure out if this is normal or something's off. I have even bought tick data from a paid service, so I am still unsure where the issue could be if there is one. Here's the report 

[Eleni Anna Branou]

ItzCayden:
I did backtest it for a year, and the results seemed incredibly unrealistic. The win ratio stayed over 90%, but the returns were overwhelming. It's my first time using an EA, so I am still a little skeptical about the returns this EA has. I still can't figure out if this is normal or something's off. I have even bought tick data from a paid service, so I am still unsure where the issue could be if there is one. Here's the report 

 I can see a lot of issues with your backtest, it uses MT4 with variable spread on a demo account, not the most realistic way of backtesting a scalping strategy.

Follow Carl's advice and put your EA on a demo or small real account to see what can really do.

[gr0spy]

The problem is that in backtester with 100% accurate tick data I get amazing results for some EAs, but in reality when taking into account all conditions it operates in EA doesn't perform like it does on 100% accurate tick data. No  matter how hard I tried, I failed every single time. I can't really say that it is fraud, but perhaps some hedge funds, investment firms and banks have access to 100% accurate data feeds and they don't want to share it with the public. I guess Citi, J.P  morgan and other institutions have access to it and average humans don't have access to such data feeds.   

[Fernando Carreiro]

gr0spy #: The problem is that in backtester with 100% accurate tick data I get amazing results for some EAs, but in reality when taking into account all conditions it operates in EA doesn't perform like it does on 100% accurate tick data. No  matter how hard I tried, I failed every single time. I can't really say that it is fraud, but perhaps some hedge funds, investment firms and banks have access to 100% accurate data feeds and they don't want to share it with the public. I guess Citi, J.P  morgan and other institutions have access to it and average humans don't have access to such data feeds.   

1. You have to consider variable spread, quote slippage, and network delays, which the Strategy Tester in MT4 does not simulate.
2. Market conditions in the future will NEVER exactly match conditions in the past. If the EA is unable to adapt dynamically and is totally dependant on optimisations, then it will tend to fail in the real world.

The moral of the story is ... the more dependant your EA is on real and accurate historical tick data, the higher the probability of it failing in real conditions.

Make your EA more adaptable and dynamic, and less dependant on optimisations, so that it can compensate for changes under real conditions.