Backtesting news trading based on economic calendar history with adjusted timings

This blogpost presents some further results of experiments with news trading based on the built-in calendar of MetaTrader 5 and MQL5.

Originally, the idea was implemented in the algotrading book as a set of classes for calendar caching and filtering, that allows for transferring the calendar records into the tester and then running backtests and optimizations driven by news.

The generation of calendar cache files (*.cal) and their replay in the tester was implemented in the indicator CalendarMonitorCached.mq5, which is now superseded by its improved modification CalendarMonitorCachedTZ.mq5 available in the codebase. Main improvement is that the timestamps of events in the history stored in the cache can now be adjusted according to (most likely changed in past) timezones of the server. Without this correction events are not synchronized in history with corresponding bars, and as a result, news backtesting is not accurate. Please find more details on the page for the indicator and also on the page of the related script CalendarCSVForDates.mq5.

In addition to the classes and indicator the book contains the example of expert adviser CalendarTrading.mq5.

Let's remind you, that the robot selects only important events with quantitative metrics and opens trades according to reported actual impact (positive/negative).

It's better to test it on a Forex major like EURUSD. With all settings by default you only need to specify the name of a cache file in CalendarFile input. This file should be generated by CalendarMonitorCachedTZ.mq5 indicator and then manually copied into the common folder for all terminals. This way you don't have a need to specify the file in the #property tester_file directive in the source code of the robot and recompile it (which would be necessary to repeat every time for another cache file).

Here we publish a slightly updated version of the expert adviser (with all dependencies in the source code) and compare 2 backtests of it - with and without corrections of times in the cache.

In other words, as the preparation for the research there were generated 2 cache files: fixed.cal and unfixed.cal. To create any type of these caches you need to provide the name of the cal-file in CalendarCacheFile input of CalendarMonitorCachedTZ indicator. For a cache without time corrections you should empty FixCachedTimesBySymbolHistory input. For a cache with corrected times (fixed) you should additionally fill in FixCachedTimesBySymbolHistory input, where you write a symbol with most reliable and complete history - XAUUSD or EURUSD (or their analogues) are recommended. The underlying lib (TimeServerDaylighSavings) will empirically deduce real timezone offsets on the history using stats of opening hours of trading weeks, and adjust calendar events accordingly.

Here are the results of tests on EURUSD,H1 for the period 01.01.2022-11.11.2024.

Backtest trading with unfixed calendar cache

Backtest trading with fixed calendar cache

They both seem not so impressive (what we'll discuss below), but the fixed calendar cache worked a little better.

And here are the fragments of the logs showing performance of trades made by specific event type.

Unfixed cache log:

     [event_id] [country] [currency] [money] [count] [pf]                             [name]
[ 0]  840200001 "US"      "USD"      -113.90     140 0.64 "EIA Crude Oil Stocks Change"     
[ 1]  840020008 "US"      "USD"       -43.71      27 0.26 "New Home Sales"                  
[ 2]  840180002 "US"      "USD"       -27.52      26 0.56 "CB Consumer Confidence Index"    
[ 3]  840120001 "US"      "USD"       -25.62      32 0.57 "Existing Home Sales"             
[ 4]  840030021 "US"      "USD"       -24.97      21 0.27 "JOLTS Job Openings"              
[ 5]  840010007 "US"      "USD"       -23.33      26 0.33 "GDP q/q"                         
[ 6]  840190001 "US"      "USD"       -21.58      33 0.39 "ADP Nonfarm Employment Change"   
[ 7]  276010008 "DE"      "EUR"       -10.94      16 0.60 "GDP q/q"                         
[ 8]  999030001 "EU"      "EUR"        -4.09       3 0.03 "Employment Change q/q"           
[ 9]  840040003 "US"      "USD"         2.98      31 1.09 "ISM Non-Manufacturing PMI"       
[10]  840020010 "US"      "USD"         4.20      32 1.09 "Retail Sales m/m"                
[11]  840020014 "US"      "USD"         5.30      24 1.26 "Core Durable Goods Orders m/m"   
[12]  840020005 "US"      "USD"         5.61      33 1.12 "Building Permits"                
[13]  276070001 "DE"      "EUR"         6.27      31 1.17 "ZEW Economic Sentiment Indicator"
[14]  250010005 "FR"      "EUR"         9.60      15 1.87 "GDP q/q"                         
[15]  999030016 "EU"      "EUR"        14.08      23 1.63 "GDP q/q"                         
[16]  840120003 "US"      "USD"        18.11      34 1.48 "Pending Home Sales m/m"          
[17]  276030003 "DE"      "EUR"        20.66      31 1.70 "Ifo Business Climate"            
[18]  840040001 "US"      "USD"        25.79      25 1.86 "ISM Manufacturing PMI"           
[19]  840030016 "US"      "USD"        36.52      35 1.77 "Nonfarm Payrolls"                
[20]  999030003 "EU"      "EUR"        48.38      33 2.69 "Retail Sales m/m"                

Fixed cache log:

     [event_id] [country] [currency] [money] [count] [pf]                             [name]
[ 0]  840200001 "US"      "USD"       -80.69     140 0.71 "EIA Crude Oil Stocks Change"     
[ 1]  840180002 "US"      "USD"       -28.07      26 0.51 "CB Consumer Confidence Index"    
[ 2]  840120001 "US"      "USD"       -15.41      32 0.68 "Existing Home Sales"             
[ 3]  840020008 "US"      "USD"       -15.34      27 0.61 "New Home Sales"                  
[ 4]  840020014 "US"      "USD"       -11.95      24 0.63 "Core Durable Goods Orders m/m"   
[ 5]  840030021 "US"      "USD"       -11.35      22 0.66 "JOLTS Job Openings"              
[ 6]  276010008 "DE"      "EUR"       -10.95      16 0.56 "GDP q/q"                         
[ 7]  276070001 "DE"      "EUR"        -3.65      31 0.92 "ZEW Economic Sentiment Indicator"
[ 8]  840120003 "US"      "USD"        -2.47      34 0.95 "Pending Home Sales m/m"          
[ 9]  999030001 "EU"      "EUR"        -1.60       3 0.10 "Employment Change q/q"           
[10]  840040003 "US"      "USD"         1.17      31 1.03 "ISM Non-Manufacturing PMI"       
[11]  999030003 "EU"      "EUR"         1.28      33 1.03 "Retail Sales m/m"                
[12]  840190001 "US"      "USD"         1.63      33 1.05 "ADP Nonfarm Employment Change"   
[13]  276030003 "DE"      "EUR"         7.61      31 1.20 "Ifo Business Climate"            
[14]  840040001 "US"      "USD"        13.72      24 1.41 "ISM Manufacturing PMI"           
[15]  840010007 "US"      "USD"        14.21      26 1.48 "GDP q/q"                         
[16]  250010005 "FR"      "EUR"        18.40      15 3.93 "GDP q/q"                         
[17]  840020010 "US"      "USD"        19.25      32 1.52 "Retail Sales m/m"                
[18]  999030016 "EU"      "EUR"        28.23      23 2.57 "GDP q/q"                         
[19]  840020005 "US"      "USD"        36.06      33 2.04 "Building Permits"                
[20]  840030016 "US"      "USD"        49.50      35 2.21 "Nonfarm Payrolls"                

The distribution of profits/losses by event type did not change a lot and can confirm more or less stable effects (even if timing is inaccurate) for events which are normally considered most impactful for the market, such as NFPs, Retail sales, PMI. With corrected timing of events one can investigate their influence on the history more thoroughly.

Now let's speculate on why the results are still not so good.

1. The expert adviser reads the field "impact_type" filled by the calendar provider to obtain trade signals, but we don't know how exactly this field is filled. For example, actual value in the news record can be technically treated as good, but it can be worse than expected, which can produce negative effect. There is a lot of space to explore for better trading signals generation utilizing more fields from the calendar.
2. From the table above it's obvious that tradable event types should be carefully selected for each specific algorithm (for example, volatility strategy would give most likely different responces for these event types), and the default markings of importance are not sufficient.
3. The expert adviser does not analyze market conditions right before each event, and it's parameters were not in any way optimized.