Significant Performance Discrepancy Between Backtesting and Live Trading - page 3

New comment
 
err3vi #: One potentially relevant detail I forgot to mention: I do not use time-based frames (M1, M5, etc.) for trading. Instead, I trade on every tick, both in backtesting and live testing. In backtesting, I specifically use the Every tick option to ensure tick-by-tick execution.

Then you are highly dependant on the quality of the tick data, and any deficiency what-so-ever in its quality will deeply impact your results.

You must in this case pay absolute attention to the logs about the tick data that the tester is recovering and verifying. Any mismatch or gap will be replaced by "virtual ticks".

So, you must absolutely ensure that the data is 100% for the time span you are testing your EA on.

 
Fernando Carreiro #:

Then you did not pay attention to my question ...

Did you check how far back, your broker offers tick data? Are there in gaps in that data? Are there mismatches between tick data and the bar OHLC data?

When there is missing real tick data or a mismatch between tick and OHLC data, then the Strategy Tester replaces it with virtual tick data which is much more "smooth" than real data.

You should be scrutinizing the logs for these failings in quality of the data.

I am not sure to understand. This is a typical log I get when starting a backtest:
CS      0       16:43:39.078    Server  MetaTester 5 started on 127.0.0.1:3000
CS      0       16:43:39.116    Startup cloud network mode is off
CS      0       16:43:39.116    Startup initialization finished
CS      0       16:43:39.564    127.0.0.1       login (build 4774)
CS      0       16:43:40.164    Network 17276 bytes of account info loaded
CS      0       16:43:40.164    Network 1478 bytes of tester parameters loaded
CS      0       16:43:40.164    Network 8900 bytes of input parameters loaded
CS      0       16:43:40.165    Network 1789 bytes of symbols list loaded (249 symbols)
CS      0       16:43:40.165    Tester  expert file added: Experts\nimbly\Virtuoso-EA.ex5. 203602 bytes loaded
CS      0       16:43:40.189    Tester  4522 Mb available, 56 blocks set for ticks generating
CS      0       16:43:40.189    Tester  initial deposit 3000.00 EUR, leverage 1:100
CS      0       16:43:40.191    Tester  successfully initialized
CS      0       16:43:40.191    Network 202 Kb of total initialization data received
CS      0       16:43:40.191    Tester  Intel Core i5-6200U  @ 2.30GHz, 12155 MB
CS      0       16:43:40.230    MQL5 debugger   connected
CS      0       16:43:40.253    Symbols FRA40: symbol to be synchronized
CS      0       16:43:40.253    Symbols FRA40: symbol synchronized, 3720 bytes of symbol info received
CS      0       16:43:40.254    History FRA40: history synchronization started
CS      0       16:43:40.256    History FRA40: load 25 bytes of history data to synchronize in 0:00:00.001
CS      0       16:43:40.256    History FRA40: history synchronized from 2021.01.01 to 2024.10.25
CS      0       16:43:40.377    History FRA40,M1: history cache allocated for 556409 bars and contains 555337 bars from 2021.01.01 01:00 to 2022.10.20 22:59
CS      0       16:43:40.377    History FRA40,M1: history begins from 2021.01.01 01:00
CS      0       16:43:40.381    Tester  FRA40,M1 (FusionMarkets-Demo): every tick generating

...

Does it answer your questions?

 

This is an log example of when tick data mismatches or there are gaps ...

And then the tester substitutes them with virtual ticks (which are smoother).


 
err3vi #: I am not sure to understand. This is a typical log I get when starting a backtest:

Does it answer your questions?

Close MetaTrader and clear your logs and any existing bar/tick data in the Testerr directory, and restart your MetaTrader and then do a single test, and look at the whole log from the beginning.

The data generated only happens once at the beginning of the test, so pay attention to what it states about the data being generated.

 
Fernando Carreiro #:

This is an log example of when tick data mismatches or there is gaps ...

And then the tester substitutes them with virtual ticks (which are smoother).

No, as far as I can recall, I have never seen these kinds of messages.
 
err3vi #: No, as far as I can recall, I have never seen these kinds of messages.
Read my previous message. Your log output shows that you are using cached data, so you won't see those messages until you re-generate the data.
 

And once more I will repeat what I and the other users have stated before ...

If your broker is providing MetaQuotes data, then it is not real data and will not match your live trading.

 
err3vi #:
The MT5 backtesting platform provides an option to use real ticks, as well as the ability to simulate slippage on historical data. By backtesting, I refer to testing the algorithm on historical data, whereas by live testing, I mean running the algorithm in real-time on a demo account.

again, you fail to read the messages. if your broker has already told you that they get data from mq, then, your tick data is not from the broker. ie backtesting is not using real ticks from the broker. Is there any more ways that we can say this?

This means that your backtesting can not be trusted -- at least your backtest results can NOT be compared with any live trading.

 
Fernando Carreiro #:
Read my previous message. Your log output shows that you are using cached data, so you won't see those messages until you re-generate the data.
This is the log I get after clearing the cache and starting a backtest in visual mode.  
GN      0       18:06:43.429    Tester  Local network farm switched off
OS      0       18:06:43.429    Tester  Cloud servers switched off
NG      0       18:07:48.627    Tester  "nimbly\Virtuoso-EA.ex5" 64 bit
HN      0       18:07:49.249    Tester  FRA40: preliminary downloading of M1 history started
IE      0       18:07:50.113    Tester  FRA40: 30% history downloaded
OL      0       18:07:50.439    Tester  FRA40: 40% history downloaded
JG      0       18:07:50.765    Tester  FRA40: 50% history downloaded
QN      0       18:07:51.087    Tester  FRA40: 60% history downloaded
HI      0       18:07:51.410    Tester  FRA40: 70% history downloaded
NP      0       18:07:51.630    Tester  FRA40: 81% history downloaded
CK      0       18:07:51.953    Tester  FRA40: 91% history downloaded
KQ      0       18:07:52.176    Tester  FRA40: preliminary downloading of M1 history completed in 0:00:02.922
LE      0       18:07:52.176    Tester  FRA40: history data begins from 2023.12.27 00:00
DP      0       18:07:52.195    Core 1  agent process started on 127.0.0.1:3000
EG      0       18:07:52.195    Core 1  connecting to 127.0.0.1:3000
FP      0       18:07:53.333    Core 1  connected
RH      0       18:07:53.364    Core 1  authorized (agent build 4774)
PN      0       18:07:53.379    Tester  FRA40,M1 (FusionMarkets-Demo): visual testing of Experts\nimbly\Virtuoso-EA.ex5 from 2025.02.26 00:00 to 2025.02.27 00:00
CP      0       18:07:54.390    Core 1  common synchronization completed
OG      3       18:07:54.546    History nothing to analyze
LF      0       18:09:37.091    Core 1  connection closed
The backtest is performed using historical data from yesterday. Yesterday was a very bad day for the live performance of the algorithm. Nonetheless, the algorithm performs exceptionally well in this test.
 
Michael Charles Schefe #:

again, you fail to read the messages. if your broker has already told you that they get data from mq, then, your tick data is not from the broker. ie backtesting is not using real ticks from the broker. Is there any more ways that we can say this?

This means that your backtesting can not be trusted -- at least your backtest results can NOT be compared with any live trading.

Yes, I understand. However, my question is: what kind of processing does MQL apply to live data before storing it as historical data? My goal is to replicate this processing in real-time to better align my live performance with backtesting results.
1234
New comment