Defining your Edge (Part 5): Using GARCH Variance and Volatility-Scaled LSTM in an Expert Advisor
91
0
Introduction
From our last-piece on developing-Your-Edge where we introduced a new algorithm-network combination as an entry signal, where we decidedly got different test results as far as using just the algorithm vs using both the algorithm and neural network. Therefore, even though we are introducing a new algorithm-network pair in this article, we slightly adjust our focus to having a more all-round algorithm that not only identifies if entry setups exist but also tries to assess if they are worth it.
Most trading approaches concentrate their efforts on choosing whether an entry setup exists or not. Not as much attention is allotted to the narrower problem of when, within a given setup, the actual entry should be taken. This distinction can become very important when price tends to move from a reasonably stable regime into one where volatility expansion is taking foothold. In other words, a breakout can be accurately spotted that still gives a poor trade when the entry is either early, late, or taken in a false expansion.
Our custom signal class for this article uses two engines. First, and primarily, we have GARCH(1,1) that approximates if near-term price volatility is increasing relative to its recent baseline. An optional Volatility-Scaled LSTM, the second engine, could then evaluate a short sequence of normalized price volatility states before contributing to the final entry likelihood. Both of these engines are controlled by seven execution modes that cover: breakouts, squeeze releases, re-entries, impulses, band walks, pullbacks, and range escapes; while utilizing: ATR-, Bollinger-Band- and price-action-based indicators.
The proposition we make is modest. GARCH could spot if a valid setup happens in a favorable setting, whereas the LSTM is engaged to help describe how this regime came about. However, the opposite can also be plausible where ATR, Bollinger Bands and GARCH overlap a lot leaving the network to add only complexity instead of valid information. In this context therefore, rapid proofing implies switching between algorithm only to algorithm plus network versions and trying to validate both with back tests and forward walks.
MQL5 Implementation
Setting the experiment as a signal class
Our custom class inherits from the 'CExpertSignal' base class and this allows the MQL5 Wizard to use it as an entry component within a broad-focused Expert Advisor. Two inputs are key in this experiment. 'ModelMode = 0', which keeps the deterministic pattern score plus GARCH filter. With 'ModelMode = 1' we retain the same structure but do blend its score with LSTM probability. 'SignalMode' independently chooses one of the seven interpretations of ATR, Bollinger and price-action.
This setting apart is important given that it prevents the neural network from becoming a second unrelated strategy. With this, both models begin with similar directional logic for their patterns, and both need to pass the GARCH requirement. They both terminate with similar 'LongCondition()' and 'ShortCondition()' voting interfaces. The network thus serves as an extra inference layer instead of a substitute for the baseline.
class CSignalGARCHLSTM : public CExpertSignal
{
protected:
CiATR m_atr;
CiBands m_bands;
int m_model_mode;
int m_signal_mode;
int m_atr_period;
int m_bands_period;
double m_bands_deviation;
ENUM_APPLIED_PRICE m_applied;
int m_pattern_lookback;
int m_garch_lookback;
double m_garch_alpha;
double m_garch_beta;
double m_garch_omega_scale;
double m_min_expansion;
double m_raw_gate;
double m_entry_threshold;
int m_forecast_horizon;
int m_lstm_sequence;
int m_lstm_hidden;
int m_lstm_samples;
int m_lstm_epochs;
double m_lstm_learning_rate;
double m_lstm_l2;
double m_lstm_weight;
//----code gap
public:
CSignalGARCHLSTM(void);
~CSignalGARCHLSTM(void) {}
void ModelMode(int value) { m_model_mode=value; }
//--- code gap---
virtual bool ValidationSettings(void);
virtual bool InitIndicators(CIndicators *indicators);
virtual int LongCondition(void);
virtual int ShortCondition(void);
protected:
bool InitATR(CIndicators *indicators);
bool InitBands(CIndicators *indicators);
double ATR(const int shift) { return(m_atr.Main(shift)); }
double BandBase(const int shift) { return(m_bands.Base(shift)); }
double BandUpper(const int shift) { return(m_bands.Upper(shift)); }
double BandLower(const int shift) { return(m_bands.Lower(shift)); }
bool ValidValue(const double value);
//--- code gap---
bool GARCHState(const int shift,double &forecast_variance,double &forecast_sigma,double &expansion_ratio); double VolatilityStrength(const int shift); double ModeScore(const int shift); double Mode0BreakoutExpansion(const int shift); double Mode1SqueezeRelease(const int shift); double Mode2BandReentry(const int shift); double Mode3MidBandImpulse(const int shift); double Mode4BandWalk(const int shift); double Mode5PullbackContinuation(const int shift); double Mode6RangeBreakout(const int shift); bool FeatureVector(const int shift,double &features[]); void InitializeLSTM(double &wx[],double &wh[],double &bias[],double &wy[],double &by); bool BuildSequence(const int shift,double &sequence[]); double LSTMForward(const double &sequence[],const double &wx[],const double &wh[],const double &bias[],const double &wy[],const double by); void TrainOneSequence(const double &sequence[],const double target,double &wx[],double &wh[],double &bias[],double &wy[],double &by); double LSTMProbability(const int shift); double ProbabilityUp(const int shift); }; The constructor we use provide starting, not optimal input values. We assign ATR period to 14, Bollinger-Bands to the typical 20/2.0, and use a 12-bar lookback window for price action. In addition, we have a 120-return GARCH window, a small LSTM with 6 sequence steps, 64 rolling samples, plus four hidden units. 'We then exclude combinations of parameters that could make the experiment harder to interpret. The most important GARCH restriction is:
Where alpha and beta are positives. Our goal here is to keep persistence below a near-unit-root region where shocks could decay so slowly that a "quick" volatility forecast becomes questionable. Furthermore, checks constrain the projection horizon, sequence length, hidden-state size, probability threshold, learning-rate as well as network weights.
//+------------------------------------------------------------------+ //| Constructor | //+------------------------------------------------------------------+ CSignalGARCHLSTM::CSignalGARCHLSTM(void) : m_model_mode(0), m_signal_mode(0), m_atr_period(14), m_bands_period(20), m_bands_deviation(2.0), m_applied(PRICE_CLOSE), m_pattern_lookback(12), m_garch_lookback(120), m_garch_alpha(0.10), m_garch_beta(0.85), m_garch_omega_scale(1.0), m_min_expansion(1.02), m_raw_gate(0.06), m_entry_threshold(0.60), m_forecast_horizon(1), m_lstm_sequence(6), m_lstm_hidden(4), m_lstm_samples(64), m_lstm_epochs(1), m_lstm_learning_rate(0.01), m_lstm_l2(0.0005), m_lstm_weight(0.35), m_cache_time(0), m_lstm_model_time(0), m_lstm_model_ready(false), m_lstm_by(0.0), m_cache_raw(0.5), m_cache_expansion(1.0), m_cache_probability(0.5) { m_used_series=USE_SERIES_OPEN+USE_SERIES_HIGH+USE_SERIES_LOW+USE_SERIES_CLOSE+USE_SERIES_TIME; } //+------------------------------------------------------------------+ //| Validate settings | //+------------------------------------------------------------------+ bool CSignalGARCHLSTM::ValidationSettings(void) { if(!CExpertSignal::ValidationSettings()) return(false); if(m_model_mode<0 || m_model_mode>1) { Print(__FUNCTION__+": ModelMode must be 0 or 1"); return(false); } if(m_signal_mode<0 || m_signal_mode>6) { Print(__FUNCTION__+": SignalMode must be from 0 through 6"); return(false); } if(m_atr_period<2 || m_bands_period<3 || m_bands_deviation<=0.0) { Print(__FUNCTION__+": invalid ATR/Bands parameters"); return(false); } if(m_pattern_lookback<3) { Print(__FUNCTION__+": PatternLookback must be at least 3"); return(false); } if(m_garch_lookback<30) { Print(__FUNCTION__+": GARCHLookback must be at least 30"); return(false); } if(m_garch_alpha<=0.0 || m_garch_beta<=0.0 || m_garch_alpha+m_garch_beta>=0.995) { Print(__FUNCTION__+": GARCH coefficients require alpha>0, beta>0 and alpha+beta<0.995"); return(false); } if(m_garch_omega_scale<=0.0) { Print(__FUNCTION__+": GARCHOmegaScale must be positive"); return(false); } if(m_min_expansion<0.80 || m_min_expansion>2.00) { Print(__FUNCTION__+": MinExpansion must be in [0.80,2.00]"); return(false); } if(m_raw_gate<0.0 || m_raw_gate>=0.50) { Print(__FUNCTION__+": RawGate must be in [0.00,0.50)"); return(false); } if(m_entry_threshold<=0.50 || m_entry_threshold>=1.0) { Print(__FUNCTION__+": EntryThreshold must be in (0.50,1.00)"); return(false); } if(m_forecast_horizon<1 || m_forecast_horizon>10) { Print(__FUNCTION__+": ForecastHorizon must be from 1 through 10"); return(false); } if(m_lstm_sequence<2 || m_lstm_sequence>32 || m_lstm_hidden<1 || m_lstm_hidden>16) { Print(__FUNCTION__+": invalid LSTM sequence or hidden width"); return(false); } if(m_lstm_samples<20 || m_lstm_epochs<1 || m_lstm_epochs>10) { Print(__FUNCTION__+": invalid LSTM sample/epoch settings"); return(false); } if(m_lstm_learning_rate<=0.0 || m_lstm_learning_rate>0.25 || m_lstm_l2<0.0) { Print(__FUNCTION__+": invalid LSTM learning settings"); return(false); } if(m_lstm_weight<0.0 || m_lstm_weight>1.0) { Print(__FUNCTION__+": LSTMWeight must be between 0 and 1"); return(false); } return(true); }
ATR and Bollinger Bands are created thanks to the Standard Library and they get registered with the signal's indicator container. The ATR gives us a range-based measure of recent activity. Bollinger for its part provides us a moving basis, a channel width plus a normalized location for price. We do run into an objection though. Both indicators are volatility sensitive, meaning GARCH could be measuring a phenomenon the baseline already sees. Instead of attempting to settle this argument theoretically, our implementation leaves it open for testing.
//+------------------------------------------------------------------+ //| Initialize base indicators, then register and create ATR/Bands. | //| Use the supplied collection and the configured symbol/timeframe. | //| Return false for a null collection | //| or any initialization failure. | //+------------------------------------------------------------------+ bool CSignalGARCHLSTM::InitIndicators(CIndicators *indicators) { if(indicators==NULL) return(false); if(!CExpertSignal::InitIndicators(indicators)) return(false); if(!InitATR(indicators)) return(false); if(!InitBands(indicators)) return(false); return(true); } //+------------------------------------------------------------------+ //| Register the member ATR indicator and create its data handle. | //| Use the current symbol/timeframe and configured ATR period. | //| Return false on null collection, registration or creation error. | //| Log registration/creation failures; no local rollback is done. | //+------------------------------------------------------------------+ bool CSignalGARCHLSTM::InitATR(CIndicators *indicators) { if(indicators==NULL) return(false); if(!indicators.Add(GetPointer(m_atr))) { Print(__FUNCTION__+": error adding ATR object"); return(false); } if(!m_atr.Create(m_symbol.Name(),m_period,m_atr_period)) { Print(__FUNCTION__+": error initializing ATR"); return(false); } return(true); } //+------------------------------------------------------------------+ //| Register and create Bollinger Bands with zero horizontal shift. | //| Use configured period, deviation, | //| applied price and chart market. | //| Return false on null collection, registration or creation error. | //| Log registration/creation failures; no local rollback is done. | //+------------------------------------------------------------------+ bool CSignalGARCHLSTM::InitBands(CIndicators *indicators) { if(indicators==NULL) return(false); if(!indicators.Add(GetPointer(m_bands))) { Print(__FUNCTION__+": error adding Bands object"); return(false); } if(!m_bands.Create(m_symbol.Name(),m_period,m_bands_period,0,m_bands_deviation,m_applied)) { Print(__FUNCTION__+": error initializing Bollinger Bands"); return(false); } return(true); }
From Market Structure to Measurements
A set of helper functions is used to translate raw prices into bounded quantities that can be mixed in a score. For these we have 'Clamp()' and 'Clamp01()' that have extremes, 'Sigmoid()' is used to map unrestricted values into probabilities, while 'SignedUnit()' helps compress signed ratios. 'CandleLocation()' gives the position of the close price within a price bar. 'BandWidth()' returns the upper-lower distance and 'BandPosition()' gives us the relative position of the close price to the Bollinger channel. Rolling ATR and band-width averages give us local reference levels, while 'RecentRange()' gives us the prior high-low boundary without having the current breakout bar re-define the range it is breaking from.
//+------------------------------------------------------------------+ //| Return true for a finite numeric value other than EMPTY_VALUE. | //| This check alone does not require a positive price or indicator. | //+------------------------------------------------------------------+ bool CSignalGARCHLSTM::ValidValue(const double value) { return(value!=EMPTY_VALUE && MathIsValidNumber(value)); } //+------------------------------------------------------------------+ //| Restrict a numeric value to the inclusive interval [lo,hi]. | //| Assume ordered bounds; no missing-value validation is performed. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::Clamp(const double value,const double lo,const double hi) { if(value<lo) return(lo); if(value>hi) return(hi); return(value); } //+------------------------------------------------------------------+ //| Restrict a numeric score to the inclusive interval [0,1]. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::Clamp01(const double value) { return(Clamp(value,0.0,1.0)); } //+------------------------------------------------------------------+ //| Map a logit to a logistic probability. | //| Clamp the input to [-30,30] before evaluating the exponential. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::Sigmoid(const double value) { double z=Clamp(value,-30.0,30.0); return(1.0/(1.0+MathExp(-z))); } //+------------------------------------------------------------------+ //| Compress a signed value using value/(1+abs(value)). | //| Return zero for magnitudes at or below 1e-12. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::SignedUnit(const double value) { double a=MathAbs(value); return(a<=1.0e-12 ? 0.0 : value/(1.0+a)); } //+------------------------------------------------------------------+ //| Locate the close within the high-low range of the shifted bar. | //| Return a score in [0,1], where 1 means a close at the high. | //| Return neutral 0.5 for invalid prices or a nonpositive range. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::CandleLocation(const int shift) { double h=High(shift),l=Low(shift),c=Close(shift); if(!ValidValue(h) || !ValidValue(l) || !ValidValue(c) || h<=l) return(0.5); return(Clamp01((c-l)/(h-l))); } //+------------------------------------------------------------------+ //| Scale the close change over lookback bars by their price range. | //| Include the shifted bar and map the change around neutral 0.5. | //| Return 0.5 for bad closes/lookback | //| or a range no larger than point. | //| Skip invalid older extremes; this helper is not called here. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::PriceMomentum(const int shift,const int lookback) { if(lookback<1) return(0.5); double c0=Close(shift),c1=Close(shift+lookback); if(!ValidValue(c0) || !ValidValue(c1)) return(0.5); double highest=High(shift),lowest=Low(shift); for(int i=1;i<=lookback;i++) { double h=High(shift+i),l=Low(shift+i); if(ValidValue(h) && h>highest) highest=h; if(ValidValue(l) && l<lowest) lowest=l; } double range=highest-lowest; if(range<=m_symbol.Point()) return(0.5); return(Clamp01(0.5+0.5*(c0-c1)/range)); } //+------------------------------------------------------------------+ //| Return upper minus lower Bollinger Band at the requested shift. | //| Return zero for missing bands or a nonpositive channel width. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::BandWidth(const int shift) { double u=BandUpper(shift),l=BandLower(shift); if(!ValidValue(u) || !ValidValue(l) || u<=l) return(0.0); return(u-l); } //+------------------------------------------------------------------+ //| Locate the close within the shifted Bollinger channel. | //| Clamp to [0,1]; closes beyond the channel saturate at its ends. | //| Return neutral 0.5 for invalid data or a nonpositive width. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::BandPosition(const int shift) { double u=BandUpper(shift),l=BandLower(shift),c=Close(shift); if(!ValidValue(u) || !ValidValue(l) || !ValidValue(c) || u<=l) return(0.5); return(Clamp01((c-l)/(u-l))); } //+------------------------------------------------------------------+ //| Average valid positive ATR values from shift over lookback bars. | //| Skip unusable samples; return zero | //| if none or lookback is below 1. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::ATRMean(const int shift,const int lookback) { if(lookback<1) return(0.0); double sum=0.0; int count=0; for(int i=0;i<lookback;i++) { double v=ATR(shift+i); if(ValidValue(v) && v>0.0) { sum+=v; count++; } } return(count>0 ? sum/count : 0.0); } //+------------------------------------------------------------------+ //| Average positive band widths from shift over lookback bars. | //| Skip unusable widths; return zero | //| if none or lookback is below 1. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::BandWidthMean(const int shift,const int lookback) { if(lookback<1) return(0.0); double sum=0.0; int count=0; for(int i=0;i<lookback;i++) { double v=BandWidth(shift+i); if(v>0.0) { sum+=v; count++; } } return(count>0 ? sum/count : 0.0); } //+------------------------------------------------------------------+ //| Write the high/low extremes of bars | //| shift+1 through shift+lookback. | //| Exclude the query bar so it can be | //| tested against the prior range. | //| Require lookback >= 2 and valid first extremes; skip later gaps. | //| Return true only for a positive | //| range; failure may alter outputs. | //+------------------------------------------------------------------+ bool CSignalGARCHLSTM::RecentRange(const int shift,const int lookback,double &highest,double &lowest) { if(lookback<2) return(false); highest=High(shift+1); lowest =Low(shift+1); if(!ValidValue(highest) || !ValidValue(lowest)) return(false); for(int i=2;i<=lookback;i++) { double h=High(shift+i),l=Low(shift+i); if(ValidValue(h) && h>highest) highest=h; if(ValidValue(l) && l<lowest) lowest=l; } return(highest>lowest); }
These helpers form a common measurement language, and are not the forecasting system. They also help make the trade decision steps more "explainable".
GARCH as a volatility gate
Our 'FeatureVector()' function begins by defining log returns:
These are then centered with a rolling mean in order to get residuals:
The average of these values is squared and it becomes the realized variance used to initialize the recursion. The persistence (p = alpha + beta), and the implementation, get the constant term as:
Once we have omega, the conditional variance gets updated from older observations up to the present:
Our resulting forecast standard deviation is (sigma_r). Entry timing is not based on that number in isolation but instead on the ratio:
When we get an expansion ratio greater than one it implies the forecast conditional volatility is above the rolling realized baseline. 'VolatilityStrength()' function then changes this relation into a bounded contribution that can be put together with directional evidence.
//+------------------------------------------------------------------+ //| GARCH(1,1) one-step variance forecast. | //| Returns forecast sigma relative to rolling realized sigma. | //+------------------------------------------------------------------+ bool CSignalGARCHLSTM::GARCHState(const int shift,double &forecast_variance,double &forecast_sigma,double &expansion_ratio) { forecast_variance=0.0; forecast_sigma=0.0; expansion_ratio=1.0; int n=m_garch_lookback; double mean=0.0; int count=0; for(int i=0;i<n;i++) { double c0=Close(shift+i),c1=Close(shift+i+1); if(!ValidValue(c0) || !ValidValue(c1) || c0<=0.0 || c1<=0.0) continue; mean+=MathLog(c0/c1); count++; } if(count<n/2 || count<20) return(false); mean/=count; double variance=0.0; for(int i=0;i<n;i++) { double c0=Close(shift+i),c1=Close(shift+i+1); if(!ValidValue(c0) || !ValidValue(c1) || c0<=0.0 || c1<=0.0) continue; double e=MathLog(c0/c1)-mean; variance+=e*e; } variance/=count; if(variance<=1.0e-16 || !MathIsValidNumber(variance)) return(false); double persistence=m_garch_alpha+m_garch_beta; double omega=(1.0-persistence)*variance*m_garch_omega_scale; if(omega<1.0e-16) omega=1.0e-16; double h=variance; for(int i=n-1;i>=0;i--) { double c0=Close(shift+i),c1=Close(shift+i+1); if(!ValidValue(c0) || !ValidValue(c1) || c0<=0.0 || c1<=0.0) continue; double e=MathLog(c0/c1)-mean; h=omega+m_garch_alpha*e*e+m_garch_beta*h; if(h<=1.0e-16 || !MathIsValidNumber(h)) h=variance; } forecast_variance=h; forecast_sigma=MathSqrt(h); expansion_ratio=forecast_sigma/MathSqrt(variance); return(ValidValue(forecast_sigma) && ValidValue(expansion_ratio)); } double CSignalGARCHLSTM::VolatilityStrength(const int shift) { double v,s,e; if(!GARCHState(shift,v,s,e)) return(0.0); return(Clamp01((e-0.90)/0.40)); }
This then defines the trading decision GARCH is meant to improve. We are not seeking whether GBPJPY should fall or rise. We are asking if a directional setup is apparent when volatility seems capable of expanding. If trades that pass this gate are no better out of sample than the same class of pattern without meaningful volatility discrimination, the proposed GARCH timing edge would have failed the basic test.
Seven different entry hypotheses
We use the function 'ModeScore()' as a dispatcher. Every mode returns a score whose median is 0.5. Bullish evidence is when it is greater than 0.5 and anything below 0.5 is bearish. Our modes are therefore 7 separate hypotheses that share a common scoring scale instead of seven votes that get averaged down.
//+------------------------------------------------------------------+ //| Select one of seven independent signal interpretations. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::ModeScore(const int shift) { switch(m_signal_mode) { case 0: return(Mode0BreakoutExpansion(shift)); case 1: return(Mode1SqueezeRelease(shift)); case 2: return(Mode2BandReentry(shift)); case 3: return(Mode3MidBandImpulse(shift)); case 4: return(Mode4BandWalk(shift)); case 5: return(Mode5PullbackContinuation(shift)); case 6: return(Mode6RangeBreakout(shift)); } return(0.5); }
Given that the output is graded, 'ModeScore()' could reject weak pattern evidence prior to the optional network being used. The LSTM thus needs to beat more than a crude Boolean baseline. It needs to add something beyond an existing confidence score.
Mode-0: Outer band breakout plus expansion
Our first operation mode probes the question of an outer Bollinger Band price break demanding immediate participation. A bullish candidate would require the close to be above the upper band and also above its open. Bearish logic would mirror this condition below the lower band. When strong we allocate 35% of this signal to the distance beyond the band, 25% to ATR rise, 20% to close location in the candle and 20% to GARCH volatility strength.
//+------------------------------------------------------------------+ //| Mode 0: close breaks an outer band as ATR and GARCH expand. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::Mode0BreakoutExpansion(const int shift) { double c=Close(shift),o=Open(shift),u=BandUpper(shift),l=BandLower(shift); double atr0=ATR(shift),atr1=ATR(shift+1),width=BandWidth(shift); if(!ValidValue(c) || !ValidValue(o) || !ValidValue(u) || !ValidValue(l) || !ValidValue(atr0) || !ValidValue(atr1) || width<=0.0 || atr1<=0.0) return(0.5); double vol=VolatilityStrength(shift); double atr_rise=Clamp01((atr0/atr1-0.95)/0.35); double bull=0.0,bear=0.0; if(c>u && c>o) bull=0.35*Clamp01((c-u)/(0.25*width)+0.20)+0.25*atr_rise+0.20*CandleLocation(shift)+0.20*vol; if(c<l && c<o) bear=0.35*Clamp01((l-c)/(0.25*width)+0.20)+0.25*atr_rise+0.20*(1.0-CandleLocation(shift))+0.20*vol; return(Clamp01(0.5+0.5*(bull-bear))); }
The decision we seek is not if we have a breakout, but if this is happening at a useful entry moment. GARCH acts like a pressure gauge and the LSTM can examine how that state is formed, and when the network adds nothing forward, band price-breaks and ATR may already have critical information we need.
Mode-1: Squeeze Release
Our second mode starts from contraction. The prior Bollinger width ought to be below 85% of its older rolling mean, following which current width and ATR should both increase strength. For weighting, the band release is allotted 35%, the ATR rise 25%, GARCH 20%, and distance from Bollinger basis 20%. The direction is got from the close-price relative to this basis.
//+------------------------------------------------------------------+ //| Mode 1: prior compression releases into widening bands. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::Mode1SqueezeRelease(const int shift) { double w0=BandWidth(shift),w1=BandWidth(shift+1); double old_mean=BandWidthMean(shift+2,m_pattern_lookback); double atr0=ATR(shift),atr1=ATR(shift+1),base=BandBase(shift),c=Close(shift); if(w0<=0.0 || w1<=0.0 || old_mean<=0.0 || !ValidValue(atr0) || !ValidValue(atr1) || !ValidValue(base) || !ValidValue(c) || atr1<=0.0) return(0.5); bool squeezed=(w1<0.85*old_mean); bool releasing=(w0>w1 && atr0>atr1); if(!squeezed || !releasing) return(0.5); double release=Clamp01((w0/w1-1.0)/0.35); double atr_rise=Clamp01((atr0/atr1-1.0)/0.30); double vol=VolatilityStrength(shift); double pos=BandPosition(shift); double strength=0.35*release+0.25*atr_rise+0.20*vol+0.20*MathAbs(2.0*pos-1.0); double direction=(c>=base ? 1.0 : -1.0); return(Clamp01(0.5+0.5*direction*strength)); }
Bands describe compression, ATR renewed range, GARCH conditional variance and LSTM the path from contraction into release. However all four may have different views of the same event. When removing the network does not change forward performance, or improve it, the sequence model will demonstrate independent value.
Mode-2: Outer band re-entry
This mode reverses the breakout idea. We define a bullish setup where the prior close is below the prior lower band and the current close is back above the lower band. The bearish logic as one would expect mirrors this by using the upper band. The score weighting assigns 35% for re-entry depth, 25% for candle location, 20% ATR relative to recent mean and 20% to GARCH strength.
//+------------------------------------------------------------------+ //| Mode 2: price returns inside a band after an outer excursion. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::Mode2BandReentry(const int shift) { double c0=Close(shift),c1=Close(shift+1); double u0=BandUpper(shift),u1=BandUpper(shift+1),l0=BandLower(shift),l1=BandLower(shift+1); double atr0=ATR(shift),atr_mean=ATRMean(shift+1,m_pattern_lookback); if(!ValidValue(c0) || !ValidValue(c1) || !ValidValue(u0) || !ValidValue(u1) || !ValidValue(l0) || !ValidValue(l1) || !ValidValue(atr0) || atr_mean<=0.0) return(0.5); double vol=VolatilityStrength(shift); double atr_state=Clamp01((atr0/atr_mean-0.80)/0.60); double bull=0.0,bear=0.0; if(c1<l1 && c0>l0) bull=0.35*Clamp01((c0-l0)/MathMax(BandWidth(shift)*0.25,m_symbol.Point()))+ 0.25*CandleLocation(shift)+0.20*atr_state+0.20*vol; if(c1>u1 && c0<u0) bear=0.35*Clamp01((u0-c0)/MathMax(BandWidth(shift)*0.25,m_symbol.Point()))+ 0.25*(1.0-CandleLocation(shift))+0.20*atr_state+0.20*vol; return(Clamp01(0.5+0.5*(bull-bear))); }
In this mode we ask whether extreme excursions are reverting or simply pausing. Often high volatility can accompany both reversal and continuation therefore recent sequences are important. From our testing below, this is the optimal mode chosen out of the seven implying the network needs to discriminate among re-entry events the algorithm often finds structurally valid. When the algorithm only version keeps stronger back and forward performance, the LSTM would not have improved this particular decision.
Mode-3: Mid balance Impulse
Our fourth mode studies Bollinger basis. A bullish event happens when price moves from at or below the prior basis to above the current basis. The bearish logic is an inverse of this and we weight this as 35% for distance beyond the basis, 25% for ATR rise, 20% for candle location, and 20% for GARCH strength.
//+------------------------------------------------------------------+ //| Mode 3: cross of the Bollinger basis with a range impulse. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::Mode3MidBandImpulse(const int shift) { double c0=Close(shift),c1=Close(shift+1),b0=BandBase(shift),b1=BandBase(shift+1); double atr0=ATR(shift),atr1=ATR(shift+1),width=BandWidth(shift); if(!ValidValue(c0) || !ValidValue(c1) || !ValidValue(b0) || !ValidValue(b1) || !ValidValue(atr0) || !ValidValue(atr1) || atr1<=0.0 || width<=0.0) return(0.5); double vol=VolatilityStrength(shift); double atr_rise=Clamp01((atr0/atr1-0.95)/0.30); double bull=0.0,bear=0.0; if(c1<=b1 && c0>b0) bull=0.35*Clamp01((c0-b0)/(0.25*width))+0.25*atr_rise+0.20*CandleLocation(shift)+0.20*vol; if(c1>=b1 && c0<b0) bear=0.35*Clamp01((b0-c0)/(0.25*width))+0.25*atr_rise+0.20*(1.0-CandleLocation(shift))+0.20*vol; return(Clamp01(0.5+0.5*(bull-bear))); }
With this mode, a middle band crossing is common so often the problem is if it represents actual impulse. GARCH and ATR often want more energetic crossings, while LSTM is able to classify the recent sequence. Confirmations can come late, however a higher win rate with worse entry price would not necessarily validate this mode.
Mode-4: Directional band walk
The fifth mode is about persistence, and its study dwells on six recent observations. Bullish evidence is when we have repeated closes above the basis when the band position is north of 0.60. The bearish evidence would take the mirror side of this with the band below 0.40. About two thirds of observations need to agree. Our weighting assigns 45% to persistence, 20% to average band position, 15% to ATR state and 20% to GARCH strength.
//+------------------------------------------------------------------+ //| Mode 4: persistent closes in one side of the volatility channel. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::Mode4BandWalk(const int shift) { int n=(m_pattern_lookback<6 ? m_pattern_lookback : 6); int bull_count=0,bear_count=0; double pos_sum=0.0; for(int i=0;i<n;i++) { double p=BandPosition(shift+i),c=Close(shift+i),b=BandBase(shift+i); if(!ValidValue(c) || !ValidValue(b)) return(0.5); pos_sum+=p; if(c>b && p>0.60) bull_count++; if(c<b && p<0.40) bear_count++; } double atr0=ATR(shift),atr_mean=ATRMean(shift+1,n); if(!ValidValue(atr0) || atr_mean<=0.0) return(0.5); double persistence_bull=(double)bull_count/n; double persistence_bear=(double)bear_count/n; double atr_state=Clamp01((atr0/atr_mean-0.85)/0.50); double vol=VolatilityStrength(shift); double mean_pos=pos_sum/n; double bull=0.0,bear=0.0; if(persistence_bull>=0.66) bull=0.45*persistence_bull+0.20*Clamp01((mean_pos-0.50)/0.40)+0.15*atr_state+0.20*vol; if(persistence_bear>=0.66) bear=0.45*persistence_bear+0.20*Clamp01((0.50-mean_pos)/0.40)+0.15*atr_state+0.20*vol; return(Clamp01(0.5+0.5*(bull-bear))); }
Ky question here is if repeated movement along one side of the channel ought to be followed rather than faded. Recurrent memory can make sense when the setup is sequential however a simple persistence count does already have memory. If it therefore performs well with GARCH, the recurrent layer would be added machinery and not an edge.
Mode-5: Pull back continuation
This mode treats the Bollinger Mid-line as a trend spine. This line, the basis, has its slope normalized by current band width as well as the prior close's distance from this basis all expressed in the same scale. We deem a setup bullish when basis has a positive slope, we had a price pullback of the close towards the basis, and price is heading back above both the basis and its prior close. The bearish logic is symmetrical. Our weighting assigns 35% to slope, 25% to pullback quality, 20% to ATR acceleration and 20% to GARCH strength.
//+------------------------------------------------------------------+ //| Mode 5: basis pullback in a trending volatility channel. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::Mode5PullbackContinuation(const int shift) { double b0=BandBase(shift),b1=BandBase(shift+1),b2=BandBase(shift+2); double c0=Close(shift),c1=Close(shift+1),w0=BandWidth(shift); double atr0=ATR(shift),atr1=ATR(shift+1); if(!ValidValue(b0) || !ValidValue(b1) || !ValidValue(b2) || !ValidValue(c0) || !ValidValue(c1) || !ValidValue(atr0) || !ValidValue(atr1) || w0<=0.0 || atr1<=0.0) return(0.5); double slope=(b0-b2)/w0; double pullback=MathAbs(c1-b1)/w0; double atr_rise=Clamp01((atr0/atr1-0.95)/0.30); double vol=VolatilityStrength(shift); double bull=0.0,bear=0.0; if(slope>0.02 && pullback<0.20 && c0>b0 && c0>c1) bull=0.35*Clamp01(slope/0.20)+0.25*Clamp01((0.20-pullback)/0.20)+0.20*atr_rise+0.20*vol; if(slope<-0.02 && pullback<0.20 && c0<b0 && c0<c1) bear=0.35*Clamp01((-slope)/0.20)+0.25*Clamp01((0.20-pullback)/0.20)+0.20*atr_rise+0.20*vol; return(Clamp01(0.5+0.5*(bull-bear))); }
In this situation the decision is when to rejoin a trend following a retracement. GARCH often demands renewed activity while LSTM helps check if a pullback is turning into a re-acceleration. Requiring extra confirmation could lead to a worse price entry, so sometimes a better-looking forecast could make the worst trading rules.
Mode-6: Recent range breakout
Our concluding mode builds a high-low range from bars prior to the signal bar. A bullish candidate requires a close above the historical high with the Bollinger Bands above 0.70; bearish rules require a close to be below the low with the position at sub 0.30. Weighting assigns 35% to distance outside the range, 20% to band-width expansion, 20% to ATR rise and 25% to GARCH strength.
//+------------------------------------------------------------------+ //| Mode 6: close breaks the recent price range while bands widen. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::Mode6RangeBreakout(const int shift) { double highest,lowest; if(!RecentRange(shift,m_pattern_lookback,highest,lowest)) return(0.5); double c=Close(shift),w0=BandWidth(shift),w1=BandWidth(shift+1); double atr0=ATR(shift),atr1=ATR(shift+1),pos=BandPosition(shift); if(!ValidValue(c) || w0<=0.0 || w1<=0.0 || !ValidValue(atr0) || !ValidValue(atr1) || atr1<=0.0) return(0.5); double band_expand=Clamp01((w0/w1-0.95)/0.35); double atr_rise=Clamp01((atr0/atr1-0.95)/0.35); double vol=VolatilityStrength(shift); double bull=0.0,bear=0.0; if(c>highest && pos>0.70) bull=0.35*Clamp01((c-highest)/(0.20*w0)+0.20)+0.20*band_expand+0.20*atr_rise+0.25*vol; if(c<lowest && pos<0.30) bear=0.35*Clamp01((lowest-c)/(0.20*w0)+0.20)+0.20*band_expand+0.20*atr_rise+0.25*vol; return(Clamp01(0.5+0.5*(bull-bear))); }
In this mode we attempt to separate genuine range escape from mere boundary probes. We can also easily make the redundancy argument here given that price is outside the range, wider bands, ATR that is on the up, and an active GARCH expansion can all be the result of the same event. If omitting one or more volatility metrics leaves forward walk results intact, then arguably the simpler implementation would be preferred.
Volatility-scaled LSTM
In using the LSTM we do not receive the selected mode score directly. Rather 'FeatureVector()' function builds six inputs: log return scaled by GARCH sigma; ATR relative to GARCH-implied price volatility; Bollinger position; Bands width relative to ATR; GARCH expansion around unity; and one-bar ATR acceleration.
//+------------------------------------------------------------------+ //| Volatility-scaled LSTM feature vector. | //| Features are bounded to approximately [-1,1]. | //+------------------------------------------------------------------+ bool CSignalGARCHLSTM::FeatureVector(const int shift,double &features[]) { ArrayResize(features,GARCHLSTM_FEATURES); double variance,sigma,expansion; if(!GARCHState(shift,variance,sigma,expansion) || sigma<=0.0) return(false); double c0=Close(shift),c1=Close(shift+1); double atr0=ATR(shift),atr1=ATR(shift+1); double width=BandWidth(shift); double pos=BandPosition(shift); if(!ValidValue(c0) || !ValidValue(c1) || c0<=0.0 || c1<=0.0 || !ValidValue(atr0) || !ValidValue(atr1) || atr0<=0.0 || atr1<=0.0 || width<=0.0) return(false); double r=MathLog(c0/c1); double sigma_price=sigma*c0; if(sigma_price<=m_symbol.Point()) sigma_price=m_symbol.Point(); features[0]=Clamp(r/(3.0*sigma),-1.0,1.0); features[1]=SignedUnit(atr0/sigma_price-1.0); features[2]=Clamp(2.0*pos-1.0,-1.0,1.0); features[3]=SignedUnit(width/(4.0*atr0)-1.0); features[4]=Clamp((expansion-1.0)/0.35,-1.0,1.0); features[5]=Clamp((atr0/atr1-1.0)/0.35,-1.0,1.0); return(true); }
This is what makes the network "volatility-scaled". Rather than learn raw GBPJPY price levels, we mostly get observations shown relative to the current volatility. Similar states can thus look similar numerically in calm periods. The weakness is equally clear as most features are derived from the same GARCH, ATR and Bollinger information already used by the algorithm. The network's strongest claim to extra value is not new information but rather the sequencing of that information in time.
LSTM state, gates and training
Our 'InitializeLSTM()' function utilizes deterministic sine- and cosine-based starting weights so that Strategy Tester runs can commence from the same state. 'BuildSequence()' then sorts six-feature observations from oldest to newest.
//+------------------------------------------------------------------+ //| Deterministic LSTM initialization for repeatable tester runs. | //+------------------------------------------------------------------+ void CSignalGARCHLSTM::InitializeLSTM(double &wx[],double &wh[],double &bias[],double &wy[],double &by) { int h=m_lstm_hidden; int f=GARCHLSTM_FEATURES; ArrayResize(wx,4*h*f); ArrayResize(wh,4*h*h); ArrayResize(bias,4*h); ArrayResize(wy,h); for(int i=0;i<ArraySize(wx);i++) wx[i]=0.10*MathSin(0.731*(i+1)); for(int i=0;i<ArraySize(wh);i++) wh[i]=0.07*MathCos(0.517*(i+1)); for(int i=0;i<ArraySize(bias);i++) bias[i]=0.0; for(int j=0;j<h;j++) bias[j]=0.50; for(int j=0;j<h;j++) wy[j]=0.10*MathSin(0.389*(j+1)); by=0.0; } bool CSignalGARCHLSTM::BuildSequence(const int shift,double &sequence[]) { int tmax=m_lstm_sequence; int f=GARCHLSTM_FEATURES; ArrayResize(sequence,tmax*f); for(int t=0;t<tmax;t++) { int src_shift=shift+(tmax-1-t); double x[]; if(!FeatureVector(src_shift,x)) return(false); for(int k=0;k<f;k++) sequence[t*f+k]=x[k]; } return(true); }
The forward pass builds the LSTM gates as:
,
,
,
,
followed by:
This last hidden state feeds an output of sigmoid that we interpret as probability of an upward move.
//+------------------------------------------------------------------+ //| LSTM forward pass. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::LSTMForward(const double &sequence[],const double &wx[],const double &wh[],const double &bias[],const double &wy[],const double by) { int hsz=m_lstm_hidden; int fsz=GARCHLSTM_FEATURES; int tmax=m_lstm_sequence; double h[],c[],hn[],cn[]; ArrayResize(h,hsz); ArrayInitialize(h,0.0); ArrayResize(c,hsz); ArrayInitialize(c,0.0); ArrayResize(hn,hsz); ArrayInitialize(hn,0.0); ArrayResize(cn,hsz); ArrayInitialize(cn,0.0); for(int t=0;t<tmax;t++) { for(int j=0;j<hsz;j++) { double zf=bias[0*hsz+j]; double zi=bias[1*hsz+j]; double zo=bias[2*hsz+j]; double zg=bias[3*hsz+j]; for(int k=0;k<fsz;k++) { double x=sequence[t*fsz+k]; zf+=wx[(0*hsz+j)*fsz+k]*x; zi+=wx[(1*hsz+j)*fsz+k]*x; zo+=wx[(2*hsz+j)*fsz+k]*x; zg+=wx[(3*hsz+j)*fsz+k]*x; } for(int k=0;k<hsz;k++) { zf+=wh[(0*hsz+j)*hsz+k]*h[k]; zi+=wh[(1*hsz+j)*hsz+k]*h[k]; zo+=wh[(2*hsz+j)*hsz+k]*h[k]; zg+=wh[(3*hsz+j)*hsz+k]*h[k]; } double gf=Sigmoid(zf); double gi=Sigmoid(zi); double go=Sigmoid(zo); double gg=MathTanh(Clamp(zg,-20.0,20.0)); cn[j]=gf*c[j]+gi*gg; hn[j]=go*MathTanh(cn[j]); } for(int j=0;j<hsz;j++) { h[j]=hn[j]; c[j]=cn[j]; } } double z=by; for(int j=0;j<hsz;j++) z+=wy[j]*h[j]; return(Clamp(Sigmoid(z),0.01,0.99)); }
The 'TrainOneSequence()' function keeps the gate, hidden and cell values needed for backpropagation through time. Error starts with 'p-target'; gradients are then worked out in a backpropagation; they get clipped to a bounded range; then regularization with L2 terms is applied via 'LSTMLearningRate'.
//+------------------------------------------------------------------+ //| One full BPTT update over one short sequence. | //+------------------------------------------------------------------+ void CSignalGARCHLSTM::TrainOneSequence(const double &sequence[],const double target,double &wx[],double &wh[],double &bias[],double &wy[],double &by) { int hsz=m_lstm_hidden; int fsz=GARCHLSTM_FEATURES; int tmax=m_lstm_sequence; double hh[],cc[],gates[]; ArrayResize(hh,(tmax+1)*hsz); ArrayInitialize(hh,0.0); ArrayResize(cc,(tmax+1)*hsz); ArrayInitialize(cc,0.0); ArrayResize(gates,tmax*4*hsz); ArrayInitialize(gates,0.0); for(int t=0;t<tmax;t++) { for(int j=0;j<hsz;j++) { double zf=bias[0*hsz+j],zi=bias[1*hsz+j],zo=bias[2*hsz+j],zg=bias[3*hsz+j]; for(int k=0;k<fsz;k++) { double x=sequence[t*fsz+k]; zf+=wx[(0*hsz+j)*fsz+k]*x; zi+=wx[(1*hsz+j)*fsz+k]*x; zo+=wx[(2*hsz+j)*fsz+k]*x; zg+=wx[(3*hsz+j)*fsz+k]*x; } for(int k=0;k<hsz;k++) { double hp=hh[t*hsz+k]; zf+=wh[(0*hsz+j)*hsz+k]*hp; zi+=wh[(1*hsz+j)*hsz+k]*hp; zo+=wh[(2*hsz+j)*hsz+k]*hp; zg+=wh[(3*hsz+j)*hsz+k]*hp; } double gf=Sigmoid(zf),gi=Sigmoid(zi),go=Sigmoid(zo),gg=MathTanh(Clamp(zg,-20.0,20.0)); gates[(t*4+0)*hsz+j]=gf; gates[(t*4+1)*hsz+j]=gi; gates[(t*4+2)*hsz+j]=go; gates[(t*4+3)*hsz+j]=gg; double cp=cc[t*hsz+j]; double cn=gf*cp+gi*gg; cc[(t+1)*hsz+j]=cn; hh[(t+1)*hsz+j]=go*MathTanh(cn); } } double z=by; for(int j=0;j<hsz;j++) z+=wy[j]*hh[tmax*hsz+j]; double p=Sigmoid(z); double dz=p-target; double gwx[],gwh[],gb[],gwy[]; ArrayResize(gwx,ArraySize(wx)); ArrayInitialize(gwx,0.0); ArrayResize(gwh,ArraySize(wh)); ArrayInitialize(gwh,0.0); ArrayResize(gb,ArraySize(bias)); ArrayInitialize(gb,0.0); ArrayResize(gwy,hsz); ArrayInitialize(gwy,0.0); for(int j=0;j<hsz;j++) gwy[j]=dz*hh[tmax*hsz+j]; double gby=dz; double dh_next[],dc_next[],dh_prev[],dc_prev[],dgates[]; ArrayResize(dh_next,hsz); ArrayInitialize(dh_next,0.0); ArrayResize(dc_next,hsz); ArrayInitialize(dc_next,0.0); ArrayResize(dh_prev,hsz); ArrayInitialize(dh_prev,0.0); ArrayResize(dc_prev,hsz); ArrayInitialize(dc_prev,0.0); ArrayResize(dgates,4*hsz); ArrayInitialize(dgates,0.0); for(int t=tmax-1;t>=0;t--) { ArrayInitialize(dh_prev,0.0); ArrayInitialize(dc_prev,0.0); ArrayInitialize(dgates,0.0); for(int j=0;j<hsz;j++) { double gf=gates[(t*4+0)*hsz+j]; double gi=gates[(t*4+1)*hsz+j]; double go=gates[(t*4+2)*hsz+j]; double gg=gates[(t*4+3)*hsz+j]; double ccur=cc[(t+1)*hsz+j]; double cprev=cc[t*hsz+j]; double tanhc=MathTanh(ccur); double dh=dh_next[j]; if(t==tmax-1) dh+=dz*wy[j]; double dgo=dh*tanhc; double dc=dc_next[j]+dh*go*(1.0-tanhc*tanhc); double dgf=dc*cprev; double dgi=dc*gg; double dgg=dc*gi; dc_prev[j]=dc*gf; dgates[0*hsz+j]=dgf*gf*(1.0-gf); dgates[1*hsz+j]=dgi*gi*(1.0-gi); dgates[2*hsz+j]=dgo*go*(1.0-go); dgates[3*hsz+j]=dgg*(1.0-gg*gg); } for(int g=0;g<4;g++) { for(int j=0;j<hsz;j++) { double d=Clamp(dgates[g*hsz+j],-5.0,5.0); gb[g*hsz+j]+=d; for(int k=0;k<fsz;k++) gwx[(g*hsz+j)*fsz+k]+=d*sequence[t*fsz+k]; for(int k=0;k<hsz;k++) { gwh[(g*hsz+j)*hsz+k]+=d*hh[t*hsz+k]; dh_prev[k]+=d*wh[(g*hsz+j)*hsz+k]; } } } for(int j=0;j<hsz;j++) { dh_next[j]=Clamp(dh_prev[j],-5.0,5.0); dc_next[j]=Clamp(dc_prev[j],-5.0,5.0); } } double lr=m_lstm_learning_rate; for(int i=0;i<ArraySize(wx);i++) wx[i]-=lr*(Clamp(gwx[i],-5.0,5.0)+m_lstm_l2*wx[i]); for(int i=0;i<ArraySize(wh);i++) wh[i]-=lr*(Clamp(gwh[i],-5.0,5.0)+m_lstm_l2*wh[i]); for(int i=0;i<ArraySize(bias);i++) bias[i]-=lr*Clamp(gb[i],-5.0,5.0); for(int j=0;j<hsz;j++) wy[j]-=lr*(Clamp(gwy[j],-5.0,5.0)+m_lstm_l2*wy[j]); by-=lr*Clamp(gby,-5.0,5.0); }
The network is intentionally small. When a compact recurrent layer cannot add stable value onto a baseline that is transparent, making it larger could increase flexibility quicker than the confidence in the underlying thesis.
Rolling labels and Wizard voting
'LSTMProbability()' function helps train past observations whose future labels are known already. For the live bar, training happens once at the beginning of a new price-bar. Later price bars only give a fresh forward pass by considering the evolution of current features. The binary target is if price post 'ForecastHorizon' bars can close above the starting close.
//+------------------------------------------------------------------+ //| Rolling, leakage-safe LSTM probability of an upward move. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::LSTMProbability(const int shift) { // For the incomplete current bar, train only once when a new bar // starts, using fully closed historical labels. Each incoming tick // then performs only a forward pass with the evolving current input. if(shift==0) { datetime stamp=Time(0); if(stamp==0) return(0.5); if(!m_lstm_model_ready || stamp!=m_lstm_model_time) { InitializeLSTM(m_lstm_wx,m_lstm_wh,m_lstm_bias,m_lstm_wy,m_lstm_by); int first=1+m_forecast_horizon; int last=first+m_lstm_samples-1; int trained=0; for(int epoch=0;epoch<m_lstm_epochs;epoch++) { for(int s=last;s>=first;s--) { double start_price=Close(s); double future_price=Close(s-m_forecast_horizon); if(!ValidValue(start_price) || !ValidValue(future_price)) continue; double seq[]; if(!BuildSequence(s,seq)) continue; double target=(future_price>start_price ? 1.0 : 0.0); TrainOneSequence(seq,target,m_lstm_wx,m_lstm_wh,m_lstm_bias,m_lstm_wy,m_lstm_by); trained++; } } m_lstm_model_ready=(trained>=m_lstm_samples/2); m_lstm_model_time=stamp; } if(!m_lstm_model_ready) return(0.5); double query[]; if(!BuildSequence(0,query)) return(0.5); return(LSTMForward(query,m_lstm_wx,m_lstm_wh,m_lstm_bias,m_lstm_wy,m_lstm_by)); } // Historical shifts are trained from a deterministic rolling window. double wx[],wh[],bias[],wy[],by; InitializeLSTM(wx,wh,bias,wy,by); int first=shift+m_forecast_horizon; int last=first+m_lstm_samples-1; int trained=0; for(int epoch=0;epoch<m_lstm_epochs;epoch++) { for(int s=last;s>=first;s--) { double start_price=Close(s); double future_price=Close(s-m_forecast_horizon); if(!ValidValue(start_price) || !ValidValue(future_price)) continue; double seq[]; if(!BuildSequence(s,seq)) continue; double target=(future_price>start_price ? 1.0 : 0.0); TrainOneSequence(seq,target,wx,wh,bias,wy,by); trained++; } } if(trained<m_lstm_samples/2) return(0.5); double query[]; if(!BuildSequence(shift,query)) return(0.5); return(LSTMForward(query,wx,wh,bias,wy,by)); }
Function 'ProbabilityUp()' then re-unites our two engines where, with 'ModelMode=0' the mode score is used alone. However under 'ModelMode=1', we blend this score with the network:
Where, (w) is the 'LSTM Weight'.
//+------------------------------------------------------------------+ //| Final upward entry score. | //+------------------------------------------------------------------+ double CSignalGARCHLSTM::ProbabilityUp(const int shift) { datetime stamp=Time(shift); // Do not bar-cache shift 0: EveryTick Wizard experts must be able // to react as the current bar and conditional variance evolve. if(shift>0 && stamp!=0 && stamp==m_cache_time) return(m_cache_probability); double raw=ModeScore(shift); double variance,sigma,expansion; if(!GARCHState(shift,variance,sigma,expansion)) return(0.5); double probability=raw; if(m_model_mode==1) { double p_lstm=LSTMProbability(shift); probability=(1.0-m_lstm_weight)*raw+m_lstm_weight*p_lstm; } probability=Clamp(probability,0.01,0.99); if(shift>0) m_cache_time=stamp; else m_cache_time=0; m_cache_raw=raw; m_cache_expansion=expansion; m_cache_probability=probability; return(probability); }
The 'LongCondition()' and 'ShortCondition()' functions then finally enforce three gates. GARCH expansion should exceed 'MinExpansion'; the raw pattern needs to clear 'RawGate'; and the last probability test needs to clear the 'EntryThreshold'. The LSTM cannot create a trade that the chosen pattern does not support. Similarly no chosen mode can bypass the volatility gate.
//+------------------------------------------------------------------+ //| Vote that price will rise. | //+------------------------------------------------------------------+ int CSignalGARCHLSTM::LongCondition(void) { int result=0; int idx=StartIndex(); double variance,sigma,expansion; if(!GARCHState(idx,variance,sigma,expansion) || expansion<m_min_expansion) return(result); double raw=ModeScore(idx); if(!ValidValue(raw) || raw<0.5+m_raw_gate) return(result); double probability=ProbabilityUp(idx); if(probability<m_entry_threshold) return(result); result=(int)MathRound(100.0*probability); return(result); } //+------------------------------------------------------------------+ //| Vote that price will fall. | //+------------------------------------------------------------------+ int CSignalGARCHLSTM::ShortCondition(void) { int result=0; int idx=StartIndex(); double variance,sigma,expansion; if(!GARCHState(idx,variance,sigma,expansion) || expansion<m_min_expansion) return(result); double raw=ModeScore(idx); if(!ValidValue(raw) || raw>0.5-m_raw_gate) return(result); double probability_down=1.0-ProbabilityUp(idx); if(probability_down<m_entry_threshold) return(result); result=(int)MathRound(100.0*probability_down); return(result); }
Our ordering maintains the baseline identifiable where any network-benefit can be seen as an incremental filtering to an existing setup. Without such a benefit, the deterministic score would remain the simpler alibi.
Post-Optimization Testing
Rapid Scanning
Optimization, arguably, can be most useful when staged. When 'ModelMode=0' is in play, the seven modes plus the shared ATR, Bollinger, and GARCH inputs get to be screened first. Only after a viable baseline seems apparent can the LSTM settings get explored. Our target would be a neighborhood of acceptable settings instead of one "exceptional-coordinate". A sharp isolated setting from optimization can suggest fragility. Below we present selected strategy runs, not "complete optimization surfaces". Thus they indicate back/forward behavior but not if the picked inputs are from a particular theme and are thus bankable. Further independent testing and diligence is necessary.
Optimization to Comparison
We test with the symbol GBPJPY on the 4-hour timeframe with a USD 10K starting deposit at 1:100 leverage. We test with and without LSTM and in both instances choose the 3rd mode, (SignalMode=2). This is the outer-band re-entry idea and its main structural settings are: ATR period 35; Bollinger Bands period 10 with deviation 2; pattern lookback of 18; GARCH lookback was 240; alpha 0.10; beta 0.73; omega scale was 1.25 Min Expansion 1.05; Raw Gate 0.12; and finally the entry threshold was 0.65.
Our network switches 'ModelMode' from 0 to 1 and uses its own active LSTM settings. These include: forecast horizon 3; sequence length 4; hidden size 2; rolling samples 128; learning rate 0.015; L2 of 0.001; and an LSTMWeight of 0.15. These LSTM-related inputs listed above are inactive when the model mode is 0. When we performed the two tests we used 'Signal_PriceLevel=28', 'Signal_TakeLevel=153.5', 'Signal_Expiration=117', and no stop loss. Our Signal thus places pending limit orders instead of executing directly at the signal tick. The reports test if the forecasting layer can get better within this wizard-built execution system. We do not seek to make a pure market-order experiment in exact-tick entry. Pending-order fill and expiration can themselves change which signals turn into trades.
In this optimization the back segment did run from 1 January 2025 through 19 January 2026. The forward walk was from 20 January 2026 through 31 July 2026. In the back test runs with the chosen "ideal" settings, we covered 1622 H4 bars and about 1.54 million ticks. The forward run was 795k ticks with overall trade counts being relatively few. As can be seen in the above posted images, algorithm-only backtest gave us USD 22K from 36 trades. If we start by going over the backtest runs of the two runs, without the LSTM the profit factor was 2.70, Sharpe Ratio 3.28, and Recovery Factor was also at 2.66. 28 of the 36 trades were profitable giving a 77.78% win rate. Maximum equity drawdown was USD 8625.41 at 22.02%.
When we enabled LSTM our results still indicated 36 trades and a similar number of losers and winners yet the net profit fell to USD 19,613, Profit Factor also fell to 2.25, Sharpe to 2.99, while Recovery Factor dropped also slightly to 2.53. Main improvement was probably reduction in equity drawdown in dollar terms to USD 7,751.17 however its percentage almost remained constant at 22.01%. The gross loss also worsened with the LSTM introduction. The order history indicates why total statistics by themselves can be misleading. For most of this backtest window both runs placed a similar number of trades and seem to have differed significantly only in October 2025.
The forward tests
The test with only the algorithm gives us USD 11,897.19 net profit from 22 trades. Profit Factor comes in at 2.88, Sharpe Ratio at 3.53, and Recovery Factor at 3.24. Eighteen trades are profitable with an 81.82% win rate. The worst equity drawdown comes in as USD 3,677.6 or 14.50%.
The algorithm plus network maintains profitability which is a good sign. However, as with the back test we are trailing the algo only baseline. Our net profit comes in at USD 10,437.71, from 21 trades with Profit Factor at 2.73, Sharpe at 3.11, and Recovery Factor at 3.02. All these are lagging. We posted an 81% win rate with a twist. Even though our equity drawdown was less in dollar terms than the baseline coming in at USD 3,454.09, when expressed in percentage terms it was slightly higher at 14.58%.
This report seems to point to an almost textbook falsification event for the LSTM advantage. The network was meant to improve discrimination among valid re-entry setups. In this forward sample it rejects at least one setup that the baseline trades successfully with the end report unable to indicate any recovery of this lost edge. To be clear, the network is not a total dud, forward Profit Factor stayed above 2.7, and the Sharpe was above of 3.0 in fact with both forward runs improving their Profit Factors when compared to the backtest runs. The narrower deduction is that the recurrent layer while maintaining viability, did not add measurable value over the Mode-2/ GARCH baseline. Below is the summary composition of these two runs:
Conclusion
Our original problem was if a valid setup could be made better by spotting a more favorable expansion in volatility. The class makes this question testable since its deterministic pattern score, GARCH gate as well as recurrent network can be handled separately instead of dealing with black box forecasts. The reports provide two distinct answers. Firstly the GARCH based Mode-2 baseline was able to live past the forward walk windows with decent profitability as well as risk-adjusted statistics. With only 36 trades in the optimization run and 22 forward test trades, our evidence is preliminary however it is encouraging enough to argue justification for more testing of the underlying re-entry-and-volatility setups.
Secondly, the LSTM seems not to improve this baseline result. We maintain profitability while slightly lowering maximum drawdown however the algorithm-only version gives more net profit and better Profit Factor, Sharpe as well as Recovery Factor in the two samples. Trade-level inspection makes this point even more clear. A delayed decision in the back test period can turn a modest loss into a much larger one. In the forward walk we also saw how the forward run filtered out a profitable short trade that was key to the baseline's good performance.
This should not mean that recurrent networks are unsuitable for volatility forecasting, but rather it may suggest that something narrower and more useful. In our configurations, sequence memory has not earned its complexity. The GARCH, ATR, Bollinger Bands and re-entry patterns could already possess sufficient information to make trade decisions. Alternatively it could be that the six LSTM features are too closely derived from what we were already using with the algorithm layer in order to give us an independent edge.
This would not count as a failed experiment for a number of reasons. Chief among them is that this is the purpose of rapid proofing. A useful development process needs to be able to toss out an attractive model combination before a larger system enshrines it. Our next question thus is not how to make the LSTM larger, but if a different mode or genuinely alternative feature can give recurrent memory something new to learn. Until we perform such a test where we clearly beat the algorithm only runs out-of-sample, the simpler model will remain a stronger explanation. It can be argued that in price-forecasting research, discipline can be more valuable than another layer of intelligence since every added component needs to be swappable, comparable, and ultimately disposable when the evidence says it should.
| name | description |
|---|---|
| 05.mq5 | Wizard Assembled Expert Advisor whose header describes used files |
| 05-SignalGARCHVolScaledLSTM.mqh | Custom Signal file needed to assemble Expert Advisor via MQL5 Wizard |
Warning: All rights to these materials are reserved by MetaQuotes Ltd. Copying or reprinting of these materials in whole or in part is prohibited.
This article was written by a user of the site and reflects their personal views. MetaQuotes Ltd is not responsible for the accuracy of the information presented, nor for any consequences resulting from the use of the solutions, strategies or recommendations described.
Designing a Multi-EA Communication Bus Using Named Pipes in MQL5
Neural Networks in Trading: The Temporal Query Model (Conclusion)
Features of Experts Advisors
The ZeroMQ Message Transfer Protocol in MQL5: Implementing the REQ/REP pattern
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use