Model Drift Watchdog: PSI-Based Auto-Retrain Alerts via a Lightweight Webhook

Introduction

A model goes stale when its live inputs move away from the distribution it was trained on, and most of us only find out from the equity curve. This article builds a lightweight MQL5 watchdog that compares a rolling sample with a captured reference using the Population Stability Index (PSI), logs every reading, and sends a webhook when the monitoring state changes.

The implementation has one PSI monitor class, a webhook client, an EMA-cross demonstration EA and a companion indicator. Two monitors track a normalized RSI and a sigmoid-transformed EMA spread as stand-ins for a model feature and a model prediction. Validation has three separate parts: independently calculated PSI cases, a EURUSD H1 backtest, and a webhook-delivery test from a live/demo chart.

What this watchdog is - and isn't

What you get is a cheap, continuous tripwire that behaves the same on any symbol, and a webhook so it can talk to a retraining job, a Telegram bot or a dashboard. The code is symbol-agnostic; the thresholds and windows are not, so expect a calibration pass on each instrument, as with any threshold-based system.

Why PSI, and not KL divergence or a Kolmogorov-Smirnov test

There are several standard ways to compare two distributions, so here is why I picked PSI.

KL divergence is the closest cousin: for two discrete distributions, PSI works out to KL(Actual‖Expected) + KL(Expected‖Actual), the sum of the divergence in both directions, also called the Jeffreys divergence. Raw KL is asymmetric (KL(P‖Q) ≠ KL(Q‖P)), which makes "how far did the live distribution move from the reference" awkward to ask, because the answer depends on which distribution you call P. Summing both directions sidesteps that without adding real complexity.

A Kolmogorov-Smirnov test asks a related but different question: are these two samples drawn from the same distribution. Its D statistic is itself a continuous distance, so you could bucket it into STABLE/WATCH/DRIFT-style bands the way this system buckets PSI. What KS doesn't hand you is an established convention for where those bands should sit; its usual role is a p-value test at a chosen significance level, so an alert built off a raw D statistic means picking cutoffs from scratch.

Those cutoffs, 0.10 and 0.25, are a heuristic from credit-risk scorecard monitoring, usually traced to Lewis (1994) and discussed in Siddiqi's Credit Risk Scorecards. A recent paper, arXiv:2307.11878, reviews that lineage and criticizes fixed cutoffs because they ignore sample size, which matters for the 50-observation live window used below. I'm not aware of a citation validating them for FX features, so treat them as where to start tuning, not where to stop.

System architecture

The signal path is short. At OnInit(), the EA pulls recent closed-bar history and passes it to CaptureReference(), which cuts it into quantile buckets. That is "normal" here only in the sense of what that block of history contained, not a verified-calm baseline. Every InpRefitCadence bars the EA reads the current feature value and the model's predicted probability, pushes both into rolling ring buffers, re-counts them against the same reference edges, computes PSI, and lets a cooldown-controlled state machine decide whether the state changed. Any state change, up or down, fires a webhook POST; a reading that stays in the same band fires nothing. Each refit also updates GlobalVariables for the companion indicator and, once at least one monitor is past warmup, writes a CSV row.

Fig. 1. Model Drift Watchdog signal path - reference capture, the rolling PSI monitor, the hysteresis state machine, and where the webhook and audit log branch off.

Two independent CPsiMonitor instances run side by side: one watches a normalized RSI as a stand-in for a feature the model consumes, the other the sigmoid of the fast/slow EMA spread as a stand-in for the model's predicted probability. The watchdog doesn't care what produces the numbers, only that they come from live price action rather than a scripted fixture.

The PSI engine

Population Stability Index compares two distributions bucket by bucket:

I pulled the formula into a standalone function rather than burying it in the monitor class, so a unit test can call the exact code the live EA runs.

WdPsiFromProportions

//+------------------------------------------------------------------+
//| WdPsiFromProportions - the actual PSI formula, standalone so a   |
//| unit-test script can call the exact same code CPsiMonitor runs   |
//+------------------------------------------------------------------+
double WdPsiFromProportions(const double &actual_props[],const double &expected_props[],int n)
  {
   double psi = 0.0;
   for(int b = 0; b < n; b++)
     {
      double a = actual_props[b];
      double e = expected_props[b];
      if(a < WD_PSI_EPSILON)
         a = WD_PSI_EPSILON;                          // empty-bucket floor, not a hard 0
      if(e < WD_PSI_EPSILON)
         e = WD_PSI_EPSILON;
      psi += (a - e) * MathLog(a / e);
     }
   return(psi);
  }

If CaptureReference() fails - typically not enough history right after attaching to a fresh chart - OnInit() prints a warning and keeps running instead of failing. IsReady() gates every later Update(), so a monitor without a reference does nothing until enough history exists and you restart it. To refuse to trade until both references are valid, return INIT_FAILED instead.

Note the epsilon floor on both sides - an empty bucket produces a division-by-zero or a MathLog(0) blow-up without it, and PSI implementations that skip this detail tend to throw silent NaNs the first time a live feed produces a genuinely rare value.

The reference is a block of history sorted and cut into equal-probability quantile buckets, not equal-width ones: equal-width buckets on a skewed feature like RSI would leave some nearly empty and inflate PSI noise before any real drift.

CPsiMonitor::CaptureReference

//+------------------------------------------------------------------+
//| CaptureReference - bin a captured window of history into         |
//| quantile edges; "reference" means "what was seen at capture      |
//| time," not a verified-calm or independently-confirmed baseline   |
//+------------------------------------------------------------------+
bool CPsiMonitor::CaptureReference(const double &history[],int history_count)
  {
   if(history_count < m_num_buckets * 5)              // need enough per bucket
      return(false);

   double sorted[];
   ArrayResize(sorted,history_count);
   for(int i = 0; i < history_count; i++)
      sorted[i] = history[i];
   SortInPlace(sorted);

   ArrayResize(m_buckets,m_num_buckets);
   double step = 1.0 / m_num_buckets;

   for(int b = 0; b < m_num_buckets; b++)
     {
      double q_lo = b * step;
      double q_hi = (b + 1) * step;
      m_buckets[b].edge_lo = (b == 0) ? -DBL_MAX : Quantile(sorted,history_count,q_lo);
      m_buckets[b].edge_hi = (b == m_num_buckets - 1) ? DBL_MAX : Quantile(sorted,history_count,q_hi);
     }

   //--- level 2: count real reference hits per bucket instead of assuming an
   //--- exact 1/N split - quantile edges only guarantee 1/N for continuous,
   //--- untied data; ties at a boundary can leave a bucket's true share off
   //--- from 1/N, especially for a low-cardinality or discrete feature
   int ref_counts[];
   ArrayResize(ref_counts,m_num_buckets);
   ArrayInitialize(ref_counts,0);
   for(int i = 0; i < history_count; i++)
     {
      double v = sorted[i];
      for(int b = 0; b < m_num_buckets; b++)
        {
         if(v > m_buckets[b].edge_lo && v <= m_buckets[b].edge_hi)
           {
            ref_counts[b]++;
            break;
           }
        }
     }
   for(int b = 0; b < m_num_buckets; b++)
      m_buckets[b].ref_prop = (double)ref_counts[b] / history_count;

   m_reference_ready = true;
   m_state           = WD_WARMUP;        // ring buffer still empty - no real reading yet
   m_cooldown_left   = 0;
   return(true);
  }

From there, Update() is what runs every refit cycle: push the new value into the ring buffer, wait until it's full at least once, re-count how many live values fall into each reference bucket, and hand those actual-vs-expected proportions to WdPsiFromProportions().

CPsiMonitor::Update

//+------------------------------------------------------------------+
//| Update - push a new value, re-bin the window, return PSI + state |
//+------------------------------------------------------------------+
EWatchdogState CPsiMonitor::Update(double new_value,double &out_psi)
  {
   out_psi = m_last_psi;
   if(!m_reference_ready)
      return(m_state);

   m_window[m_window_head] = new_value;
   m_window_head = (m_window_head + 1) % m_window_size;
   if(m_window_count < m_window_size)
      m_window_count++;

   if(m_window_count < m_window_size)                 // still filling the ring
      return(m_state);

   //--- level 2: count actual hits per reference bucket
   int counts[];
   ArrayResize(counts,m_num_buckets);
   ArrayInitialize(counts,0);

   for(int i = 0; i < m_window_count; i++)
     {
      double v = m_window[i];
      for(int b = 0; b < m_num_buckets; b++)
        {
         if(v > m_buckets[b].edge_lo && v <= m_buckets[b].edge_hi)
           {
            counts[b]++;
            break;
           }
        }
     }

   //--- level 2: PSI_i = (actual - expected) * ln(actual / expected)
   double actual_props[], expected_props[];
   ArrayResize(actual_props,m_num_buckets);
   ArrayResize(expected_props,m_num_buckets);
   for(int b = 0; b < m_num_buckets; b++)
     {
      actual_props[b]   = (double)counts[b] / m_window_count;
      expected_props[b] = m_buckets[b].ref_prop;
     }
   double psi = WdPsiFromProportions(actual_props,expected_props,m_num_buckets);

   m_last_psi = psi;
   out_psi    = psi;

   //--- level 2: hysteresis - skip state re-evaluation while cooldown is active
   if(m_cooldown_left > 0)
      return(m_state);

   EWatchdogState target = WD_STABLE;
   if(psi >= m_drift_line)
      target = WD_DRIFT;
   else
      if(psi >= m_watch_line)
         target = WD_WATCH;

   if(target != m_state)
     {
      m_state         = target;
      m_cooldown_left = m_cooldown_bars;
     }

   return(m_state);
  }

The countdown lives in its own method, called once per bar and separate from Update(), which runs only every InpRefitCadence bars. Decrementing inside Update() would make InpCooldownBars scale with the cadence: 15 bars at a cadence of 5 would freeze the state for about 75.

CPsiMonitor::TickCooldown

//+------------------------------------------------------------------+
//| TickCooldown - decrement cooldown once per bar, independent of   |
//| refit cadence, so InpCooldownBars counts bars, not refit cycles  |
//+------------------------------------------------------------------+
void CPsiMonitor::TickCooldown(void)
  {
   if(m_cooldown_left > 0)
      m_cooldown_left--;
  }

A note on quantile binning and sample size

CaptureReference() refuses to run if history_count is under five times the bucket count, a floor of 50 bars with the default 10 buckets; InpRefHistoryBars defaults to 500. Quantile edges from too few points are unstable: a 10-bucket split of 12 points puts about one point per bucket, and one outlier at capture time can distort "expected" for the life of the reference. Five per bucket is a minimum, not a target. On a liquid pair at H1 or below, use several hundred bars.

The first and last edges are pinned to -DBL_MAX and DBL_MAX rather than the sample's min and max, so a live value more extreme than anything in the reference still lands in a bucket. Without that, a genuine tail event - exactly what you want PSI to notice - could fall outside every bucket and understate drift when it matters most. The expected share of each bucket is counted from the reference data, not assumed to be 1/N, which is only exact when no values are tied at an edge.

CaptureReference() can't know whether its history was calm. If the market was mid-transition during that window, the reference encodes a moving target and every later reading measures against it. Choosing a trustworthy reference window is a judgment this system can't make for you.

Cooldown-controlled state transitions

A raw PSI value hovering around 0.25 would flip a naive alert on and off every few bars and train you to ignore the webhook. The fix is a cooldown rather than a confirmation window: after a transition, the state machine stays frozen for InpCooldownBars bars, even if PSI swings across a line in between. TickCooldown() counts down once per bar, so the length stays in real bars whatever InpRefitCadence is. This is not classic hysteresis with separate entry and exit thresholds; both directions use the same two lines.

Once the cooldown clears, the next refit that lands past a threshold moves the state immediately, with no requirement that several readings agree. That is a deliberate simplification: easy to unit-test, at the cost of being a little trigger-happy right after a cooldown expires.

The cooldown length is a trade-off. Too short, and a PSI hovering at a threshold re-triggers the machine every few bars, spamming alternating WATCH/DRIFT webhooks with no new information. Too long, and a fast drift event waits behind a cooldown started by an earlier transition.

The default is 15 bars, which is 15 hours on H1. The feature monitor in the real run (Fig. 2) still flipped states more often than I'd like from 2026.03.19 onward, so a longer cooldown or a wider gap between the two lines is worth testing before live use.

Wiring it into a live EA

OnInit() builds both monitors with the same bucket count, window size and thresholds, captures both references, creates the EA's MQL5\Files subfolder (FileOpen() won't create it implicitly), records the reference window and bucket edges in reference_<magic>.csv, and opens the audit CSV. Two details matter. GlobalVariable names include the symbol and timeframe as well as the magic number, so a reused magic on two charts can't overwrite the same four variables. And the audit CSV opens read-write with a seek to the end, writing its header only for a new file, because plain FILE_WRITE truncates on every restart and erases the earlier audit trail.

OnInit

//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
  {
   g_handle_rsi      = iRSI(_Symbol,_Period,InpRsiPeriod,PRICE_CLOSE);
   g_handle_ema_fast = iMA(_Symbol,_Period,InpEmaFastPeriod,0,MODE_EMA,PRICE_CLOSE);
   g_handle_ema_slow = iMA(_Symbol,_Period,InpEmaSlowPeriod,0,MODE_EMA,PRICE_CLOSE);

   if(g_handle_rsi == INVALID_HANDLE || g_handle_ema_fast == INVALID_HANDLE ||
      g_handle_ema_slow == INVALID_HANDLE)
     {
      Print("DriftWatchdog: indicator handle creation failed");
      return(INIT_FAILED);
     }

   g_feature_monitor.Configure(InpNumBuckets,InpWindowSize,InpCooldownBars,
                                InpWatchLine,InpDriftLine);
   g_prediction_monitor.Configure(InpNumBuckets,InpWindowSize,InpCooldownBars,
                                   InpWatchLine,InpDriftLine);

   if(!WdCaptureFeatureReference())
      Print("DriftWatchdog: feature reference capture failed, not enough history yet");
   if(!WdCapturePredictionReference())
      Print("DriftWatchdog: prediction reference capture failed, not enough history yet");

   if(!WdEnsureFilesFolder(WD_SUBFOLDER))
      Print("DriftWatchdog: FolderCreate failed, err=",GetLastError());

   WdWriteReferenceLog();

   //--- level 1: scope the GlobalVariable names to symbol+period too, not just
   //--- the magic number - two charts sharing a magic number would otherwise
   //--- silently overwrite each other's PSI readings
   g_var_prefix = "WD_" + IntegerToString(InpMagicNumber) + "_" + _Symbol +
                  "_" + EnumToString((ENUM_TIMEFRAMES)_Period) + "_";

   string path = WD_SUBFOLDER + "\\audit_" + IntegerToString(InpMagicNumber) + ".csv";
   bool   appending = FileIsExist(path);
   g_file_handle = FileOpen(path,FILE_READ|FILE_WRITE|FILE_CSV|FILE_ANSI,',');
   if(g_file_handle == INVALID_HANDLE)
      Print("DriftWatchdog: FileOpen failed, err=",GetLastError());
   else
     {
      //--- level 2: FILE_WRITE alone truncates on every restart, which throws
      //--- away every prior session's audit trail - reopen for read+write and
      //--- seek to the end instead, only writing the header on a fresh file
      FileSeek(g_file_handle,0,SEEK_END);
      if(!appending)
         FileWrite(g_file_handle,"time","feature_psi","feature_state","pred_psi","pred_state");
     }

   GlobalVariableSet(g_var_prefix + "FeaturePsi",0.0);
   GlobalVariableSet(g_var_prefix + "PredPsi",0.0);
   GlobalVariableSet(g_var_prefix + "FeatureState",0.0);
   GlobalVariableSet(g_var_prefix + "PredState",0.0);

   return(INIT_SUCCEEDED);
  }

OnTick() gates everything on a new-bar check, so the refit cadence counts bars rather than ticks. The watchdog runs alongside an order-placing EMA-cross demo strategy; its risk-management implementation is outside this article's scope.

WdTryDemoTrade() runs before the refit-and-webhook block, so the webhook cannot hold up that bar's order. A synchronous WebRequest() does block the handler, though, so a slow endpoint can delay the rest of that tick's processing and any events queued behind it.

OnTick

//+------------------------------------------------------------------+
//| Expert tick function                                             |
//+------------------------------------------------------------------+
void OnTick()
  {
   datetime bar_time = iTime(_Symbol,_Period,0);
   if(bar_time == g_last_bar_time)
      return;                                          // refit/trade logic is per-bar, not per-tick
   g_last_bar_time = bar_time;
   g_bar_counter++;

   double spread_unused;
   double prob = WdModelProbability(spread_unused);

   WdTryDemoTrade(prob);

   //--- cooldown ticks down every bar, independent of refit cadence
   g_feature_monitor.TickCooldown();
   g_prediction_monitor.TickCooldown();

   if(g_bar_counter % InpRefitCadence != 0)
      return;

   double feature_val = WdNormalizedRsi();
   double feature_psi;
   EWatchdogState feature_prev = g_feature_monitor.State();
   EWatchdogState feature_now  = g_feature_monitor.Update(feature_val,feature_psi);

   double pred_psi;
   EWatchdogState pred_prev = g_prediction_monitor.State();
   EWatchdogState pred_now  = g_prediction_monitor.Update(prob,pred_psi);

   GlobalVariableSet(g_var_prefix + "FeaturePsi",feature_psi);
   GlobalVariableSet(g_var_prefix + "PredPsi",pred_psi);
   GlobalVariableSet(g_var_prefix + "FeatureState",(double)feature_now);
   GlobalVariableSet(g_var_prefix + "PredState",(double)pred_now);

   //--- level 2: each monitor is judged on its own state - a monitor in WARMUP
   //--- never alerts, and it never blocks the other monitor either
   if(feature_prev != WD_WARMUP && feature_now != WD_WARMUP)
      WdFireWebhookOnTransition("feature_rsi",feature_prev,feature_now,feature_psi);
   if(pred_prev != WD_WARMUP && pred_now != WD_WARMUP)
      WdFireWebhookOnTransition("prediction_prob",pred_prev,pred_now,pred_psi);

   //--- level 2: skip the row only while both monitors are still warming up
   if(feature_now == WD_WARMUP && pred_now == WD_WARMUP)
      return;

   WdLogAuditRow(feature_psi,feature_now,pred_psi,pred_now);
  }

Each monitor's previous state is captured before Update(), so WdFireWebhookOnTransition() can compare old and new and act only on a change. The two monitors are judged separately: one still in WARMUP never alerts and never holds the other back, and its first real reading is not a transition because there is no earlier real state to compare with. The audit row is skipped only while both monitors are warming up; if one is ready, the row is written with an empty PSI field and the state WARMUP for the other:

WdFireWebhookOnTransition

//+------------------------------------------------------------------+
//| WdFireWebhookOnTransition - only posts when state actually moved |
//+------------------------------------------------------------------+
void WdFireWebhookOnTransition(string tag,EWatchdogState prev,EWatchdogState now,double psi)
  {
   if(prev == now)
      return;

   CWebhookClient client;
   client.Configure(InpWebhookUrl,InpWebhookTimeoutMs);
   string payload = WdBuildJsonPayload(tag,_Symbol,InpMagicNumber,psi,now,TimeCurrent());

   bool ok = client.PostJson(payload);
   if(!ok)
      PrintFormat("DriftWatchdog: webhook POST failed for %s (%s -> %s): %s",
                  tag,WdStateToString(prev),WdStateToString(now),client.LastError());
   else
      PrintFormat("DriftWatchdog: webhook delivered for %s (%s -> %s), PSI=%.4f",
                  tag,WdStateToString(prev),WdStateToString(now),psi);
  }

OnTester() returns the larger of the two monitors' last PSI values so batches of runs can be ranked in the Optimization Results tab. A sum would mix distributions on different scales; the max reads plainly as "how bad is the worse one".

OnTester

//+------------------------------------------------------------------+
//| Tester statistics hook - surface both PSI readings post-run      |
//+------------------------------------------------------------------+
double OnTester()
  {
   PrintFormat("DriftWatchdog OnTester: feature_psi=%.4f (%s), pred_psi=%.4f (%s)",
               g_feature_monitor.LastPsi(),WdStateToString(g_feature_monitor.State()),
               g_prediction_monitor.LastPsi(),WdStateToString(g_prediction_monitor.State()));
   //--- level 2: the two PSI series come off different distributions and
   //--- different scales, so summing them would be a meaningless composite;
   //--- the max is at least an honest "worse of the two" sort key for
   //--- ranking optimizer passes, not a normalized joint statistic
   return(MathMax(g_feature_monitor.LastPsi(),g_prediction_monitor.LastPsi()));
  }

The webhook layer

The webhook client is intentionally thin - one WebRequest() call, no DLL, no external library. InpWebhookUrl has to be whitelisted in Tools > Options > Expert Advisors > "Allow WebRequest for listed URL" before any of this works; skip that step and every POST fails with GetLastError()=4014 before it even leaves the terminal.

CWebhookClient::PostJson

//+------------------------------------------------------------------+
//| PostJson - blocking POST via WebRequest, never throws to caller  |
//+------------------------------------------------------------------+
bool CWebhookClient::PostJson(string json_body)
  {
   if(StringLen(m_url) == 0)
     {
      m_last_error = "webhook URL not set";
      return(false);
     }

   //--- level 1: WebRequest() and StringToCharArray() both take char[],
   //--- not uchar[] - the wrong element type is a real compile risk
   char   post_bytes[];
   char   result_bytes[];
   string result_headers;
   string headers = "Content-Type: application/json\r\n";

   int post_len = StringToCharArray(json_body,post_bytes,0,WHOLE_ARRAY,CP_UTF8) - 1;
   ArrayResize(post_bytes,MathMax(0,post_len));

   ResetLastError();
   int res = WebRequest("POST",m_url,headers,m_timeout_ms,post_bytes,result_bytes,result_headers);

   if(res == -1)
     {
      m_last_error = StringFormat("WebRequest failed, GetLastError=%d (check URL is in "
                                   "Tools>Options>Expert Advisors allowed list)",GetLastError());
      return(false);
     }

   if(res < 200 || res >= 300)
     {
      m_last_error = StringFormat("webhook returned HTTP %d",res);
      return(false);
     }

   m_last_error = "";
   return(true);
  }

The payload is a flat, six-field JSON object built by hand rather than through a JSON library, since the shape never varies: tag, symbol, magic number, PSI value, new state, timestamp. A fixed shape doesn't make the fields safe to drop into a template, though. symbol comes straight from _Symbol, and a broker-specific name with a stray quote or backslash would produce invalid JSON, so WdJsonEscape() escapes both characters before the template is filled:

WdBuildJsonPayload

//+------------------------------------------------------------------+
//| WdJsonEscape - escape backslashes and double-quotes so a symbol  |
//| or tag containing either doesn't corrupt the JSON shape          |
//+------------------------------------------------------------------+
string WdJsonEscape(string s)
  {
   StringReplace(s,"\\","\\\\");
   StringReplace(s,"\"","\\\"");
   return(s);
  }

//+------------------------------------------------------------------+
//| WdBuildJsonPayload - hand-rolled, no library needed for this     |
//| flat shape; tag/symbol are escaped since callers pass _Symbol    |
//| and free-text tags, neither of which is guaranteed quote-free    |
//+------------------------------------------------------------------+
string WdBuildJsonPayload(string tag,string symbol,long magic,double psi,
                           EWatchdogState state,datetime ts)
  {
   return(StringFormat(
          "{\"tag\":\"%s\",\"symbol\":\"%s\",\"magic\":%I64d,"
          "\"psi\":%.5f,\"state\":\"%s\",\"time\":\"%s\"}",
          WdJsonEscape(tag),WdJsonEscape(symbol),magic,psi,WdStateToString(state),
          TimeToString(ts,TIME_DATE|TIME_SECONDS)));
  }

Point InpWebhookUrl at whatever your retraining pipeline, alerting bot or dashboard expects and the EA does the rest.

Testing the math on its own, before trusting it live

Before putting this on a chart, I wanted proof the formula was right independent of the EA, the ring buffer and any market data. A standalone script calls WdPsiFromProportions() against three cases worked out by hand beforehand: a moderate 5-bucket shift, a 10-bucket case that drives one bucket to zero to exercise the epsilon floor, and identical distributions, which must return exactly zero.

DriftWatchdog_PsiUnitTest.mq5

//+------------------------------------------------------------------+
//|                                  [DriftWatchdog_PsiUnitTest.mq5] |
//|                       https://www.mql5.com                       |
//+------------------------------------------------------------------+
#property copyright "Olamide Adebayo"
#property link      "https://www.mql5.com"
#property version   "1.00"
#property script_show_inputs

#include <ModelDriftWatchdog/DriftWatchdogLib.mqh>

#define WD_TEST_TOLERANCE 0.000001                   // float compare slack

//+------------------------------------------------------------------+
//| WdRunCase - compares WdPsiFromProportions() against a value      |
//| worked out independently (by hand / in Python), not by the EA    |
//+------------------------------------------------------------------+
bool WdRunCase(string name,double &actual[],double &expected[],int n,double known_psi)
  {
   double got = WdPsiFromProportions(actual,expected,n);
   double diff = MathAbs(got - known_psi);
   bool   pass = (diff <= WD_TEST_TOLERANCE);

   PrintFormat("[%s] %s - got=%.8f expected=%.8f diff=%.8f",
               pass ? "PASS" : "FAIL",name,got,known_psi,diff);
   return(pass);
  }

//+------------------------------------------------------------------+
//| Script program start function                                    |
//+------------------------------------------------------------------+
void OnStart()
  {
   int total = 0, passed = 0;

   //--- level 1: moderate-shift case, 5 even buckets vs a real skew
   double exp1[] = {0.20,0.20,0.20,0.20,0.20};
   double act1[] = {0.25,0.22,0.18,0.15,0.20};
   total++;
   if(WdRunCase("moderate_shift_5bucket",act1,exp1,5,0.02955469509754257))
      passed++;

   //--- level 1: empty-bucket case, exercises the WD_PSI_EPSILON floor
   double exp2[] = {0.10,0.10,0.10,0.10,0.10,0.10,0.10,0.10,0.10,0.10};
   double act2[] = {0.00,0.05,0.08,0.09,0.10,0.12,0.13,0.14,0.15,0.14};
   total++;
   if(WdRunCase("empty_bucket_10bucket",act2,exp2,10,0.7889669809861843))
      passed++;

   //--- level 1: identical distributions must give exactly zero
   double exp3[] = {0.25,0.25,0.25,0.25};
   double act3[] = {0.25,0.25,0.25,0.25};
   total++;
   if(WdRunCase("identical_distribution_zero",act3,exp3,4,0.0))
      passed++;

   PrintFormat("DriftWatchdog PSI unit test: %d/%d cases passed",passed,total);
   if(passed == total)
      Print("DriftWatchdog PSI unit test: ALL CASES PASSED");
   else
      Print("DriftWatchdog PSI unit test: FAILURES PRESENT, see above");
  }

Run as a script from the Navigator, this printed:

What real drift actually looks like

With the math verified, the question is whether the monitor catches a real PSI shift on an actual chart. I ran the full EA on EURUSD H1 with InpWindowSize=50 and InpRefitCadence=5, so the ring buffer needs 250 bars to first fill.

InpProbSteepness=4.0 sets where the demo strategy's 0.60/0.40 entry thresholds fall (about a 101-point fast-minus-slow EMA spread at 4.0). It was not tuned for this run. The sigmoid is monotonic and the buckets are quantile-based, so rescaling barely moves PSI; I checked that on synthetic data only (identical to four decimals for steepness 1 to 16), not on the EURUSD run.

The reference does not come from the tester range. At OnInit() the EA reads the 500 closed H1 bars before the first tester bar: 2026.01.29 22:00 to 2026.02.27 22:00 server time on MetaQuotes-Demo, from the terminal's local history for that server. The 478 bars of the tester range alone therefore cannot reproduce the numbers below. The EA prints the window to the Journal and writes it, with both monitors' bucket edges and expected shares, to reference_990071.csv in the archive. Every bucket holds exactly 50 of 500 values (10.0%) in this run, so no ties skewed the split.

Here's the real audit trail from one such run, 2026.03.16 through 2026.03.27:

Fig. 2. Live PSI trail from a real EURUSD H1 run, straight from the EA's own audit CSV. Both monitors jump straight into DRIFT at the first real reading, 2026.03.16 09:00, and never produce a STABLE reading for the rest of the test.

While the ring buffer fills, both monitors sit in WARMUP. They share a window size and cadence, so they leave it at the same refit, and no row is written and no webhook fires before then. The first row, 2026.03.16 09:00:00, is already DRIFT for both monitors; no transition through WATCH was observed. From there the feature monitor oscillates between WATCH and DRIFT while the prediction monitor stays in DRIFT.

Here are the first rows of audit_990071.csv from that run, unedited:

Across the 46 logged refits, the feature monitor spent 19 readings in WATCH and 27 in DRIFT, and the prediction monitor logged 46 DRIFT readings with a peak PSI of 2.04843. Neither monitor produced a STABLE reading once PSI became real. The cooldown counts bars, and the trail shows it: after the feature monitor drops to WATCH at 2026.03.25 17:00, a 0.28945 reading at 2026.03.26 03:00 is held at WATCH, and the 08:00 reading, 15 bars after the change, is the first one allowed to move the state back to DRIFT.

I tried two separate periods on this symbol and timeframe as a quiet control, and both showed sustained drift. Neither tested period provided a validated stable control. These runs therefore demonstrate PSI changes, but do not establish a false-alarm rate or validate the default thresholds. PSI also can't say whether the shift is a genuine regime change, and this article has no external confirmation. The tuning guidance below was calibrated against a drifting window, so treat it as a starting point for your own instrument. A less reactive feature than a raw RSI and an EMA-spread sigmoid should show longer stable stretches; test that on your own inputs.

Tuning notes: warmup and window size

Update() returns without computing until the ring buffer has filled once, and the refit runs only every InpRefitCadence bars, so the first PSI reading needs InpWindowSize × InpRefitCadence bars. My first settings, 200 and 10, needed 2,000 bars and gave a flat 0.00000 on a shorter test, which looked like a false-negative bug until I did the multiplication. With 50 and 5 the minimum is 250 bars, about ten trading days of H1, and that produced Fig. 2. Work out that product before your first run.

Cadence also changes what the window means. The live window is 50 observations taken every 5 bars, so it spans about 250 bars; it is not the last 50 consecutive bars. Changing the cadence changes both the time span and the sample composition, not just the computation load. Pick a window large enough that one refit's bucket counts aren't dominated by noise (the same five-per-bucket logic as reference capture), then the smallest cadence you can afford.

Fifty observations over 10 buckets average about five per bucket, the same floor CaptureReference() enforces on the reference side, but here it is the steady-state operating point. That is thin: sampling noise can swing a single refit's bucket counts, and with the epsilon floor an empty bucket can add a lot to PSI. The values above 1.0 and 2.0 in Fig. 2 were observed, but without a control estimate I did not measure how much of them comes from the small sample. Widening the window or cutting the bucket count steadies the reading, at the cost of a slower monitor.

Proving the webhook fires - on a live chart, not in the tester

WebRequest() calls made inside the Strategy Tester run in a sandbox with no real network access, even with the URL whitelisted. That keeps backtests reproducible, but it means a tester run can prove the PSI math and the state transitions, never webhook delivery.

To test delivery, a second small script builds a sample payload and calls the same CWebhookClient::PostJson() the live EA uses. Run it from a real or demo chart:

DriftWatchdog_WebhookTest.mq5

//+------------------------------------------------------------------+
//|                              [DriftWatchdog_WebhookTest.mq5]     |
//|                       https://www.mql5.com                       |
//+------------------------------------------------------------------+
#property copyright "Olamide Adebayo"
#property link      "https://www.mql5.com"
#property version   "1.00"
#property script_show_inputs

#include <ModelDriftWatchdog/DriftWatchdogLib.mqh>

//--- level 1: inputs so the URL can be changed without recompiling
input string InpWebhookUrl     = "http://10.136.192.17:8000/"; // listener address
input int    InpWebhookTimeout = 3000;                         // ms

//+------------------------------------------------------------------+
//| Script program start function                                    |
//|                                                                  |
//| This runs on a live/demo chart, NOT in the Strategy Tester -     |
//| WebRequest() is sandboxed away from real network access inside   |
//| the tester even when the URL is whitelisted, so a tester run     |
//| can never prove webhook delivery. A script attached to a real    |
//| chart uses the terminal's actual network stack, so this is the   |
//| honest way to test the URL, firewall, and listener together.     |
//+------------------------------------------------------------------+
void OnStart()
  {
   Print("DriftWatchdog webhook test: starting, target=",InpWebhookUrl);

   CWebhookClient client;
   client.Configure(InpWebhookUrl,InpWebhookTimeout);

   string payload = WdBuildJsonPayload("webhook_connectivity_test",_Symbol,
                                        0,0.0,WD_STABLE,TimeCurrent());

   bool ok = client.PostJson(payload);

   if(ok)
      Print("DriftWatchdog webhook test: DELIVERED - check webhook_log.txt "
            "on the listener machine for this payload: ",payload);
   else
      Print("DriftWatchdog webhook test: FAILED - ",client.LastError());
  }

On the receiving end, a small standard-library Python listener (http.server and socketserver) logged every POST with a timestamp to the console and a file. Run from the same machine, the script printed DELIVERED in the Experts tab and the listener caught the payload:

The script verifies payload construction and HTTP delivery through the same client used by the EA: the exact WdBuildJsonPayload()/CWebhookClient::PostJson() path, sent through WebRequest(), past a whitelisted URL and the OS firewall, to a listener. It does not verify an actual monitor transition followed by delivery from the running EA. WdFireWebhookOnTransition() calls the identical PostJson(), so the code is shared, but shared code is not a separately observed trigger-to-delivery test.

One troubleshooting note: WebRequest() can refuse a loopback address with error 4014 even when the URL is whitelisted. If the listener is on the same machine and 127.0.0.1 fails, use the machine's LAN IP and confirm that exact URL is in the whitelist.

Reading it day to day

The EA provides two local views of the same calculation. The audit CSV (audit_<magic>.csv) holds every post-warmup refit's PSI and state for both monitors, which is how Fig. 2 was built with no manual transcription. The companion DriftWatchdog_ProbView indicator plots the latest values, published through terminal GlobalVariables, with the watch and drift lines in a sub-window. The CSV keeps the full history, while webhooks report only state transitions, and a failed POST does not affect either local output. Because every post-warmup refit writes a row whether or not the state changed, the CSV also records the WATCH-band readings that never became an alert.

Extending this beyond a single feature

Nothing ties the class to two instances. Scaling to five or ten features means an array of monitors, a loop over CaptureReference() and Update(), and a feature name in each payload. Correlated features drifting together can produce several near-simultaneous POSTs for one underlying shift, so de-duplicate on the receiving end. A wider fan-out also means a noisier webhook stream during any real regime change, a load question the two-monitor setup never faced.

The payload is generic JSON, so anything can consume it: a retraining job, or Zapier or Make routing to Telegram or Discord so a human decides. The human-in-the-loop route is often the sensible one until you trust your threshold tuning enough to let an alert act on its own.

Conclusion

The implementation combines reference capture, rolling PSI calculation, cooldown-controlled states, local logging and webhook delivery. The formula passed three independent test cases; the EURUSD H1 run recorded sustained PSI shifts; and a separate live/demo-chart test verified HTTP delivery. False-alarm rates and transition-to-delivery behavior in the running EA remain unvalidated.

To adapt the watchdog, replace WdNormalizedRsi() and WdModelProbability() with your own feature and prediction, set InpWebhookUrl to your endpoint, and calibrate the reference window, refit cadence, bucket count, thresholds and cooldown against your own data.

Files