How different are backtest results against demo against live?

 
I have been developing an EA that takes many days to open a position but when it does it makes cash.

I wanted to ask how were your experiences of backtesting vs demo vs live environment? Are achieved results similar?
 
No.
 
Marco vd Heijden:
No.
Why would them be different?

 
Because they are not the same.
 
Marco vd Heijden:
Because they are not the same.
LOL. Very helpful.
 
Marco vd Heijden:
Because they are not the same.

/facepalm

The demo accounts and backtests usually (not all, but you never know) use "perfect" situations and information that you might or might not ever see when actually trading a live account.  Stuff like spread variation, more or less slippage on one day or another (when the trading price drops or jumps huge amounts at once).  Usually, backtesting doesn't take these into consideration as they happen, they usually just see the aftereffects.

 
Alain Verleyen:
LOL. Very helpful.

haha well i had already taken the time to send an detailed explanation to Oscar Ortiz by Private Mssg.



 
Marco vd Heijden:

haha well i had already taken the time to send an detailed explanation to Oscar Ortiz by Private Mssg.



Can I ask why in private ? It will be good if we had a serious topic about this, as it's a recurring question people ask. There are some general advice as the one of JD4 which are correct, but a detailed explanation would be very welcome. Personally I don't have time to do it.
 

It is because we now have a new messaging system that allows real time communication i think this is a great update and it is a lot faster then posting to the board i use it a lot for fast talks about several subjects, and helping friends that have problem coding something not everybody wants their code to be out in the open so it is a great and fast update.

Mr. Oscar Ortiz did not provide which terminal he is using.

Also this is a topic that is covered already many times i believe we have spoken about it last week or two weeks ago.

There is a good article about the tester, but it deals with MetaTrader 3, MetaTrader 4, and MetaTrader 5 , at the same time.

So i have highlighted as follows,

MetaTrader 3, 

MetaTrader 4

MetaTrader 5

A little history about the strategy tester


MetaTrader 3

The first strategy tester initially appeared in the MetaTrader 3 terminal. This was a relatively simple tester by modern standards; it performed testing based on three models of prices development at the bar:
  • Model of four prices - the price consecutively went through stages of Open, Low, High and Close for the bullish candle, and for the bearish candle it went through Open, High, Low and Close;
  • The model "Every 1 point" - uses a wave model 3-5-3, in which the price consecutively goes through a three-wave, then a five-wave, and then another three-wave, with an increment of 1 point;
  • Model "Spread / 2" - uses the same model as "Every one point", but the price increment is half of the spread, indicated by the user.

Price modeling has always been based on the bars of the tested time frame, information from lower time frames were not used. The tester from the MetaTrader 3 terminal had many shortcomings, including slow testing speed, low accuracy, and the lack of Expert Advisor optimization by input parameters.

MetaTrader 4

The MetaTrader 4 terminal replaced the third terminal and included a new compiled language MQL4 (Previous MQL-II was interpreted), as well as took an absolutely new approach to testing. Now testing could be carried out in three modes:

  • Every tick (Every Tick) - generation of ticks within the candle, allows for a maximum close modeling of Expert Advisor's work to real trading, within the testing environment;
  • Checkpoints (Control points) - a compromise between accuracy and speed of testing;
  • By opening prices (Open Price) - the launch of the Expert Advisor is made only at the opening of the candle, this allows for a very rapid assessment of the strategy.

An important difference from the tester of the third terminal was that the testing strategy of the MetaTrader 4 terminal used the price data of the youngest available time frame for trading simulation.

Therefore, with the presence of the minute history, throughout the tested interval, the test results are maximally close to the results obtained online.

In the absence of the underlying time frames, simulation of price development within the bar is generated in the same way as in the tester terminal of MetaTrader 3.

In addition, we gained an opportunity to perform optimization by direct enumeration of input parameters, as well as using a genetic algorithm. This gave us an opportunity to significantly accelerate the process of optimization, especially for strategies with a large number of input parameters. It is now possible to conduct the actual testing in the visual regime. This was a huge step, which was appreciated by traders.

The new MetaTrader terminal of the 5-th generation is based on the experience gained from the design of the previous terminals, and this applies to the strategy tester. Now there is an opportunity to conduct testing of multi-currency strategies, i.e. strategies which are simultaneously traded on multiple instruments.

Strategy optimization can now be conducted not only on all available processor cores, but also on remote agents, located on other computers in the LAN and the global Internet network. This allows you to build up the powers of the tester and perform cloud calculations, which were previously inaccessible, directly in the language of MQL5.

But for a basic understanding of the testing process in the MetaTrader 5 terminal, it is critical to be able to understand how the modeling of prices in a strategy tester takes place.


Algorithm of ticks' generation

Strategy Tester of the MetaTrader 5 terminal uses only one mode of price modeling in testing - the generation of ticks on the basis of existing historical data on minute time frames of the used symbols.

The remaining modes of simulations in MetaTrader 4 were removed because despite their high speed, they failed to provide a high accuracy of testing.

Using an M1 time frame in the tester, allows for a very accurate simulation of the price movement, with a minimum number of errors, in contrast to the simulation of ticks based on senior time frames. As a result, the errors in the modeling of prices in the MetaTrader 5 strategy tester are trivial, and the differences between the simulated price and the price that took place in reality, can only be within the scale of a minute bar.

The ability to perform optimization on local and remote agents, in this approach, can compensate for the increase in testing time. The generation of ticks is based on the cached minute entries in an integer format. Therefore, the generation of ticks is made very quickly.

The bars of all the required time frames are formed in the historical data base of the tester in the usual way (just like in the client terminal) with the receiving of the generated ticks. The minute bar tick volume 1 is not subjected to any generation - it can be written with a value of Close.
MQL4: automated forex trading, strategy tester and custom indicators with MetaTrader
  • www.mql4.com
MQL4: automated forex trading, strategy tester and custom indicators with MetaTrader
 

So it depends on which terminal you use, what time frame you use, and the resolution and completeness of the historic data.

Personally, i only use the strategy tester to see if the logic is working correctly, and nothing more that that.

The exact Algorithm is in the article, too.

https://www.mql5.com/en/articles/75

 
Marco vd Heijden:

So it depends on which terminal you use, what time frame you use, and the resolution and completeness of the historic data.

Personally, i only use the strategy tester to see if the logic is working correctly, and nothing more that that.

The exact Algorithm is in the article, too.

https://www.mql5.com/en/articles/75

Thank you, I also use the Strategy Tester only for testing logic and debugging purpose.

But a lot of people believe they can use for other purpose, which can be more or less true depending of the strategy. As JD4 said a real account has to deal with spread variation, slippage/requote, but also eventually loss of connection, low liquidity, etc...I would like, a day, to create a topic not only to tell these things but to demonstrate concretely the actual impact of all these parameters on the trading results.

Reason: