Random Matrix Theory: Denoising the Correlation Matrix for Multi-Symbol EAs

Introduction

If you've ever built a multi-symbol Expert Advisor that leans on correlation — a basket allocator, a pairs filter, a hedge-ratio calculator, anything that reads a correlation matrix and turns it into position sizes — you've probably noticed the numbers move around more than they should. If you run the same correlation estimate on Monday's and Tuesday's closes with identical parameters, the matrix can still shift meaningfully—even when nothing structural has changed. That's not a bug in your code. It's a property of the matrix itself.

A sample correlation matrix estimated from a finite window of returns is a noisy object. With N symbols and a lookback of T bars, if T isn't comfortably larger than N, most of what you're measuring is sampling error, not genuine co-movement. Feed that noisy matrix straight into a portfolio variance calculation or a basket sizing formula, and you get outputs that look precise — six decimal places, clean-looking weights — while actually being dominated by estimation noise. The precision is fake.

This article works through Random Matrix Theory (RMT) as a practical fix for that problem, implemented natively in MQL5 with no ALGLIB and no external DLLs. Specifically, we use the Marchenko–Pastur distribution to separate signal eigenvalues from noise eigenvalues. We then rebuild a cleaned matrix that preserves the signal structure and flattens the rest. We'll walk through the math, the native Jacobi eigendecomposition that makes this possible without external libraries, the denoising procedure itself, and a basket Expert Advisor that uses the cleaned matrix to size positions across four correlated FX/metals instruments.

Scope note: this article focuses on correlation matrix denoising as a risk-input preprocessing step. It is not a full portfolio optimizer, and the basket EA's position sizing is intentionally simplified for clarity — margin-aware sizing, transaction cost modeling, and multi-account considerations are called out explicitly in the edge cases section rather than baked in silently.

Why raw correlation matrices are mostly noise

Start with the basic statistics problem. To estimate an N x N correlation matrix you need to estimate N(N-1)/2 pairwise correlations from T observations. As N grows relative to T, you're asking a limited amount of data to pin down a rapidly growing number of parameters. Classical statistics tells you the individual correlation estimates get noisier; what's less obvious is what happens to the matrix as a whole once you look at its eigenvalues.

A true, noise-free correlation matrix describing a market with genuine common factors — a market-wide risk factor, a metals-vs-FX split, a dollar factor — has an eigenvalue spectrum with a handful of large eigenvalues (the real factors) and the rest clustered near 1.0 (idiosyncratic, symbol-specific variance). But a sample correlation matrix estimated from finite data doesn't reproduce that spectrum cleanly. Even if you fed it pure random noise with zero true correlation, the sample matrix's eigenvalues would spread out into a predictable, non-trivial distribution purely from sampling error. That distribution has a name and a closed form, and it's the key to everything that follows.

This matters for MQL5 development specifically because basket EAs, hedge-ratio calculators, and portfolio-level risk sizing routines are common enough in the Expert Advisor world, and almost all of the ones I've seen feed a raw sample correlation matrix directly into the sizing logic. The instability isn't visible in a single backtest run — it shows up as weight sets that swing between runs with slightly different date ranges, or live sizing that whipsaws between rebalances even when the underlying market relationships haven't actually changed.

There's also a subtler failure mode worth naming: overfitting to noise dimensions is a real risk in anything that optimizes against a correlation matrix, not just an aesthetic complaint. A hedge ratio or a variance-minimizing weight vector derived from a raw sample matrix can latch onto a spurious correlation that exists only in that particular lookback window, and then get invalidated the moment the window rolls forward. Denoising doesn't eliminate this risk entirely, but it removes the eigenvalue directions that are provably indistinguishable from noise before anything downstream gets a chance to overfit to them.

Random Matrix Theory and the Marchenko-Pastur distribution

Random Matrix Theory studies the statistical properties of matrices whose entries come from random processes. The specific result we need here is the Marchenko-Pastur (MP) theorem, which describes the limiting eigenvalue distribution of a correlation (or covariance) matrix estimated from T independent observations of N variables, under the null hypothesis that there is no true correlation structure at all — pure noise.

The MP distribution is parameterized by the ratio Q = T/N. As Q increases (more data relative to the number of variables), the noise distribution narrows and concentrates around 1.0, matching the intuition that more data gives cleaner estimates. As Q approaches 1 (T close to N), the distribution spreads out dramatically — this is exactly the regime where naive correlation estimation breaks down.

The MP distribution has a closed-form upper edge:

lambda_max = sigma^2 * (1 + 1/sqrt(Q))^2

where Q = T / N and sigma^2 is the variance of the underlying series (1.0 for a correlation matrix by construction). In the MQL5 code below, sqrt(Q) is computed with the native MathSqrt() function — same operation, MQL5 syntax.

The practical use of this bound is simple: compute the eigenvalues of your real sample correlation matrix, and compare each one against lambda_max. Any eigenvalue that falls at or below the MP upper edge is statistically indistinguishable from what pure noise would have produced at that same T and N — so we have no reasonable basis to treat it as a real market factor. Eigenvalues above the edge are extremely unlikely to arise from noise alone; they represent genuine, persistent common structure — a broad market factor, a metals-cluster factor, whatever the dominant co-movement in your basket happens to be.

On the theoretical assumptions: the Marchenko-Pastur derivation formally assumes independent, identically distributed observations drawn from a distribution with finite variance. Market returns satisfy none of this cleanly — consecutive bars are not independent (volatility clusters), tails are fatter than Gaussian, and the underlying correlation structure itself drifts over time (heteroscedasticity). None of that invalidates using the MP edge in practice; it means this is a practical RMT-cleaning heuristic borrowed from a theoretical result, not an exact statistical test with guaranteed coverage. Treat lambda_max as a well-motivated, principled threshold rather than a hard statistical boundary with a precise p-value attached to it.

Eigenvalue spectrum against the Marchenko-Pastur upper edge

Fig. 1. Eigenvalue spectrum of a synthetic 12-symbol correlation matrix (T=250, Q≈20.8) against the theoretical Marchenko-Pastur upper edge. Bars above the dashed line are treated as genuine common factors; bars below it are flattened as noise.

Two things are worth noting about this figure. First, this particular illustration uses a wider 12-symbol synthetic basket rather than the 4-symbol live basket in the EA, purely because a 4-eigenvalue spectrum doesn't show the bulk-versus-signal split clearly — the concept generalizes down to any N, including our small live basket, but it's easier to see with more eigenvalues on screen. Second, notice how few eigenvalues actually clear the bar. In this synthetic example with two genuine underlying factors, only two eigenvalues sit above lambda_max — the rest, despite looking like meaningful numbers in the raw matrix, are noise by the MP test. All synthetic figures are controlled illustrations of specific mechanics. The EA trades a real 4‑symbol basket, and the figures are not representative of its measured results; those are reported in the testing section.

One small but practical detail in the classification rule: the code below tests eigenvalues[i] > lambda_max (strict inequality), not >=. This matters only in the edge case of an eigenvalue landing almost exactly on the boundary, where it makes noise the default outcome rather than signal — a conservative choice, since misclassifying a genuine factor as noise merely shrinks it toward the average, while misclassifying noise as signal preserves an eigenvalue that shouldn't be trusted at all. We did not add an epsilon band around lambda_max. For the basket sizes targeted here, lambda_max varies more from sample to sample (as Q changes) than eigenvalues that happen to lie close to the theoretical edge. A wider basket with eigenvalues clustered tightly near lambda_max would be a reasonable place to revisit this with an explicit epsilon-band or a soft/continuous shrinkage function instead of a hard cutoff.

Native Jacobi eigendecomposition in MQL5

Denoising requires an eigendecomposition of a real, symmetric N x N matrix — every correlation matrix is symmetric by construction, and this restriction is what makes the classic cyclic Jacobi eigenvalue algorithm a good fit. It's iterative, numerically stable for symmetric matrices, doesn't require an external linear algebra library, and is simple enough to implement natively in MQL5 without excessive code.

The Jacobi method repeatedly applies plane rotations to zero out off-diagonal elements. Each sweep picks pairs (p, q) and rotates the matrix by an angle chosen specifically to eliminate the (p,q) and (q,p) entries. After enough sweeps, the off-diagonal mass shrinks toward zero and the diagonal converges to the eigenvalues, while the accumulated product of rotations gives you the eigenvectors.

1. Initialize V as the identity matrix (it will accumulate the eigenvectors).
2. For each sweep, scan all (p, q) pairs with p < q.
3. For each pair, compute the rotation angle phi = 0.5 * atan2(2*a[p][q], a[q][q] - a[p][p]).
4. Apply the rotation to rows/columns p and q of the working matrix, and to columns p and q of V.
5. Repeat sweeps until the sum of squared off-diagonal elements drops below a tolerance, or a sweep cap is hit.
6. Read eigenvalues off the diagonal; eigenvectors are the columns of V.

Because MQL5 only allows the first dimension of an array to be dynamic, a genuine N x N matrix with a runtime-determined N can't be declared as double matrix[][] directly. The CSquareMatrix class in this project works around that by storing the matrix as a flat 1-D buffer with manual row-major indexing — Get(i,j) and Set(i,j,v) compute i*N+j under the hood. Here is the complete class, exactly as used by every routine downstream of it:

//+------------------------------------------------------------------+
//| Flat-storage square matrix (row-major, 1-D buffer)               |
//+------------------------------------------------------------------+
class CSquareMatrix
  {
public:
    int      N;
    double   Data[];

    void    Init(const int n)
      {
       N = n;
       ArrayResize(Data, N*N);
       ArrayInitialize(Data, 0.0);
      }

    double   Get(const int i, const int j) const { return Data[i*N+j]; }
    void    Set(const int i, const int j, const double v) { Data[i*N+j] = v; }

    void    SetIdentity()
      {
       ArrayInitialize(Data, 0.0);
       for(int i=0; i<N; i++)
          Set(i, i, 1.0);
      }

    void    Copy(const CSquareMatrix &src)
      {
       Init(src.N);
       ArrayCopy(Data, src.Data);
      }
  };

With the matrix type in place, here is the complete JacobiEigenDecomposition() function — the full sweep loop, convergence test, diagonal update, and eigenvector accumulation, not an excerpt:

//+------------------------------------------------------------------+
//| Cyclic Jacobi eigenvalue decomposition for a real symmetric      |
//| matrix. Native implementation - no ALGLIB, no external DLLs.     |
//| Eigenvalues come back on the diagonal of the rotated matrix;     |
//| eigenvectors accumulate as columns of V.                         |
//+------------------------------------------------------------------+
bool JacobiEigenDecomposition(const CSquareMatrix &A_in,
                               double &eigenvalues[],
                               CSquareMatrix &V,
                               const int max_sweeps = 100,
                               const double tol = 1.0e-12)
  {
   int n = A_in.N;
   if(n <= 0)
      return false;

   CSquareMatrix A;
   A.Copy(A_in);

   V.Init(n);
   V.SetIdentity();

   ArrayResize(eigenvalues, n);

//--- iterate sweeps until the off-diagonal mass converges below tolerance
   for(int sweep=0; sweep<max_sweeps; sweep++)
     {
      //--- convergence test: sum of squared off-diagonal elements
      double off = 0.0;
      for(int p=0; p<n; p++)
         for(int q=p+1; q<n; q++)
            off += A.Get(p,q)*A.Get(p,q);

      if(off < tol)
         break;

      //--- sweep through every (p,q) pair, applying a Jacobi rotation that zeroes A(p,q)
      for(int p=0; p<n-1; p++)
        {
         for(int q=p+1; q<n; q++)
           {
            double apq = A.Get(p,q);
            if(MathAbs(apq) < 1.0e-15)
               continue;

            double app = A.Get(p,p);
            double aqq = A.Get(q,q);

            //--- rotation angle that zeroes A(p,q) and A(q,p)
            double phi = 0.5*MathArctan2(2.0*apq, aqq-app);
            double c   = MathCos(phi);
            double s   = MathSin(phi);

            //--- rotate columns p, q of the working matrix
            for(int k=0; k<n; k++)
              {
               double akp = A.Get(k,p);
               double akq = A.Get(k,q);
               A.Set(k,p, c*akp - s*akq);
               A.Set(k,q, s*akp + c*akq);
              }
            //--- rotate rows p, q of the working matrix
            for(int k=0; k<n; k++)
              {
               double apk = A.Get(p,k);
               double aqk = A.Get(q,k);
               A.Set(p,k, c*apk - s*aqk);
               A.Set(q,k, s*apk + c*aqk);
              }
            //--- accumulate the same rotation into V (eigenvectors)
            for(int k=0; k<n; k++)
              {
               double vkp = V.Get(k,p);
               double vkq = V.Get(k,q);
               V.Set(k,p, c*vkp - s*vkq);
               V.Set(k,q, s*vkp + c*vkq);
              }
           }
        }
     }

//--- eigenvalues now sit on the diagonal of the fully rotated matrix
   for(int i=0; i<n; i++)
      eigenvalues[i] = A.Get(i,i);

   return true;
  }

//+------------------------------------------------------------------+
//| Sort eigenvalues descending, reordering eigenvector columns      |
//+------------------------------------------------------------------+
void SortEigenDescending(double &eigenvalues[], CSquareMatrix &V)
  {
   int n = ArraySize(eigenvalues);
   for(int i=0; i<n-1; i++)
     {
      int max_idx = i;
      for(int j=i+1; j<n; j++)
         if(eigenvalues[j] > eigenvalues[max_idx])
            max_idx = j;
      if(max_idx != i)
        {
         double tmp = eigenvalues[i];
         eigenvalues[i] = eigenvalues[max_idx];
         eigenvalues[max_idx] = tmp;

         //--- swap the matching eigenvector columns so V stays aligned with eigenvalues
         for(int k=0; k<n; k++)
           {
            double t2 = V.Get(k,i);
            V.Set(k,i, V.Get(k,max_idx));
            V.Set(k,max_idx, t2);
           }
        }
     }
  }

For the basket sizes this article targets — 4 to maybe 15 symbols — a Jacobi sweep converges in well under 100 iterations (5-6 sweeps in practice for the 4- and 12-symbol matrices used in this article's own validation run below), and each sweep is O(N^3) at worst, which runs comfortably inside a rebalance call that fires every 20 bars rather than every tick. This isn't the algorithm you'd reach for on a 500-symbol universe, but for a realistic MetaTrader 5 multi-symbol basket it's more than adequate, and it keeps the entire pipeline dependency-free.

The denoising algorithm, step by step

With eigendecomposition available, the denoising procedure itself is short:

1. Build the T x N log-return matrix for the basket, aligned on the anchor symbol's bar timestamps.
2. Compute the sample Pearson correlation matrix C_raw from the return matrix.
3. Eigendecompose C_raw via the native Jacobi routine, sort eigenvalues descending.
4. Compute lambda_max from Q = T/N.
5. Classify: eigenvalues above lambda_max are kept unchanged; eigenvalues at or below it are replaced by their common average.
6. Reconstruct C_denoised = V * diag(shrunk eigenvalues) * V^T.
7. Re-symmetrize for floating point safety and force the diagonal back to exactly 1.0.

Before denoising, the correlation matrix itself has to be built from returns. Here is the complete BuildCorrelationMatrix() function, including the degenerate-variance guard mentioned later in the edge cases section:

//+------------------------------------------------------------------+
//| Pearson correlation matrix from a T x N return matrix            |
//+------------------------------------------------------------------+
void BuildCorrelationMatrix(const CRectMatrix &R, CSquareMatrix &C)
  {
   int T = R.Rows;
   int N = R.Cols;
   C.Init(N);

   double mean[], stdev[];
   ArrayResize(mean, N);
   ArrayResize(stdev, N);

//--- pass 1: per-symbol mean return over the window
   for(int j=0; j<N; j++)
     {
      double s = 0.0;
      for(int t=0; t<T; t++)
         s += R.Get(t,j);
      mean[j] = s/T;
     }

//--- pass 2: per-symbol standard deviation, floored to avoid a division by zero below
   for(int j=0; j<N; j++)
     {
      double ss = 0.0;
      for(int t=0; t<T; t++)
        {
         double d = R.Get(t,j) - mean[j];
         ss += d*d;
        }
      stdev[j] = MathSqrt(ss/(T-1));
      if(stdev[j] < 1.0e-12)
         stdev[j] = 1.0e-12; // guard a flat/zero-variance series
     }

//--- pass 3: pairwise Pearson correlation from the covariance and the two stdevs
   for(int i=0; i<N; i++)
     {
      for(int j=0; j<N; j++)
        {
         double cov = 0.0;
         for(int t=0; t<T; t++)
            cov += (R.Get(t,i)-mean[i]) * (R.Get(t,j)-mean[j]);
         cov /= (T-1);
         C.Set(i,j, cov/(stdev[i]*stdev[j]));
        }
     }

//--- diagonal is exactly 1.0 by definition, regardless of any rounding above
   for(int i=0; i<N; i++)
      C.Set(i,i,1.0);
  }

Step 5 deserves a closer look, because there's a design choice buried in it. Rather than simply zeroing out the noise eigenvalues, they're replaced with their average. This preserves the trace of the matrix — the sum of all eigenvalues, which for a correlation matrix equals N, the total variance in the system. If you zero the noise eigenvalues outright, you shrink the total variance and the reconstructed matrix stops being a valid correlation matrix (diagonal entries drift away from 1.0 in a way that's harder to repair cleanly). Averaging keeps the total variance budget intact and only redistributes how much of it gets attributed to "real" versus "noise" directions.

And here is the complete DenoiseCorrelationRMT() function that ties the pieces above together — the Q-ratio guard, the eigendecomposition call, the classification, the trace-preserving reconstruction, and the re-symmetrization pass, in full:

//+------------------------------------------------------------------+
//| Marchenko-Pastur denoising of a correlation matrix. Eigenvalues  |
//| at or below the MP upper edge are averaged and replaced          |
//| (trace-preserving, so total variance is unchanged); eigenvalues  |
//| above the edge are treated as genuine common factors and kept    |
//| untouched.                                                       |
//+------------------------------------------------------------------+
bool DenoiseCorrelationRMT(const CSquareMatrix &C_raw, const int T,
                            CSquareMatrix &C_denoised,
                            double &lambda_max_out,
                            int &n_signal_out, int &n_noise_out)
  {
   int N = C_raw.N;
//--- Q must stay well above 1, or the Marchenko-Pastur edge becomes meaningless
   if(T <= N)
     {
      Print("RMT: T <= N, Marchenko-Pastur Q ratio collapses - refusing to denoise.");
      return false;
     }

//--- theoretical MP upper edge: eigenvalues at or below this are indistinguishable from noise
   double Q = (double)T / (double)N;
   double sigma2 = 1.0; // correlation matrix: mean eigenvalue is 1 by construction
   double lambda_max = sigma2 * MathPow(1.0 + 1.0/MathSqrt(Q), 2.0);
   lambda_max_out = lambda_max;

//--- eigendecompose, then sort descending so the largest (most likely genuine) eigenvalues come first
   double eigenvalues[];
   CSquareMatrix V;
   if(!JacobiEigenDecomposition(C_raw, eigenvalues, V))
      return false;

   SortEigenDescending(eigenvalues, V);

//--- classify each eigenvalue against lambda_max and accumulate the noise average
   int n_signal = 0, n_noise = 0;
   double noise_sum = 0.0;

   for(int i=0; i<N; i++)
     {
      if(eigenvalues[i] > lambda_max)
         n_signal++;
      else
        {
         noise_sum += eigenvalues[i];
         n_noise++;
        }
     }

//--- trace-preserving shrinkage: replace every noise eigenvalue with their shared average
   double noise_avg = (n_noise > 0) ? noise_sum/n_noise : 0.0;

   double denoised_eig[];
   ArrayResize(denoised_eig, N);
   for(int i=0; i<N; i++)
      denoised_eig[i] = (eigenvalues[i] > lambda_max) ? eigenvalues[i] : noise_avg;

   ReconstructFromEigen(denoised_eig, V, C_denoised);

//--- re-symmetrize for floating point safety, then force the diagonal back to exactly 1.0
   int n = C_denoised.N;
   for(int i=0; i<n; i++)
     {
      for(int j=i; j<n; j++)
        {
         double avg = (C_denoised.Get(i,j) + C_denoised.Get(j,i)) / 2.0;
         C_denoised.Set(i,j, avg);
         C_denoised.Set(j,i, avg);
        }
     }
   for(int i=0; i<n; i++)
      C_denoised.Set(i,i, 1.0);

   n_signal_out = n_signal;
   n_noise_out  = n_noise;
   return true;
  }

Raw correlation matrix (12-symbol synthetic basket)

Fig. 2a. Raw sample correlation matrix for the same 12-symbol synthetic basket used in Fig. 1.

Denoised correlation matrix (same basket, after Marchenko-Pastur cleaning)

Fig. 2b. The same matrix after Marchenko-Pastur denoising. The weaker, noise-driven off-diagonal entries visibly compress toward a common shrinkage level while the strong factor-driven blocks (S1-S6, S7-S9) stay distinguishable.

One consequence of this that's easy to miss until you see it in a chart: the denoised matrix isn't "smoother-looking" in some vague aesthetic sense — the individual noise-driven correlations genuinely become more similar to each other, because they're all being pulled toward the same noise-eigenvalue average. Real, factor-driven relationships stay distinguishable from each other; the fake ones collapse toward a shared baseline. That's the whole point.

The heatmap makes the effect visible at a glance, but seeing the actual numbers move is often more convincing. Here is one real 4x4 correlation matrix — same 4-symbol basket size and T=250 lookback the live EA uses — before and after denoising, taken from the same validation run behind the table in the next section:

Raw C:

1.00  0.41  0.22  0.12
0.41  1.00  0.20  0.13
0.22  0.20  1.00  0.26
0.12  0.13  0.26  1.00

Denoised C:

1.00  0.28  0.25  0.20
0.28  1.00  0.25  0.19
0.25  0.25  1.00  0.17
0.20  0.19  0.17  1.00

The strongest raw correlation (0.41, symbols 1-2) survives denoising as the strongest denoised correlation (0.28) — shrunk, not erased, because it's part of what the single signal eigenvalue in this window is capturing. The weaker raw entries (0.12, 0.13) move up toward that same shrinkage band instead of staying anchored to their noisier original values — the "noise pulled toward a common average" effect from the heatmap, in actual numbers.

On positive semi-definiteness: reconstruction from non‑negative eigenvalues preserves PSD algebraically. However, the subsequent re-symmetrization and diagonal-forcing steps can perturb the matrix and are not guaranteed to preserve PSD. This was checked directly rather than assumed: the fully post-processed 4x4 matrix above has a minimum eigenvalue of 0.72 — comfortably positive — and a stress test across 500 random synthetic baskets from N=4 to N=12 produced zero cases of a negative eigenvalue after full post-processing. That's not a formal proof for every possible input — a basket with eigenvalues clustered very close to zero before denoising could in principle be more fragile — and adding an explicit nearest-correlation-matrix repair step is a reasonable defensive addition for a much larger or more degenerate basket. For the sizes and data tested here, it wasn't necessary.

To check whether the denoising step actually buys anything in practice beyond looking cleaner, the EA logs a Frobenius-norm distance between the current correlation matrix and the previous rebalance's matrix, for both the raw and denoised versions, on every cycle. If denoising is doing its job, that distance should be consistently smaller for the denoised series — meaning the matrix used for sizing decisions isn't jumping around from one rebalance to the next just because of estimation noise.

Rolling Frobenius-norm stability, raw vs. denoised

Fig. 3. Frobenius norm distance between consecutive rebalance windows, raw versus denoised correlation matrix, across 45 rolling windows on the logged basket. The denoised series is both lower on average and less volatile window-to-window.

This is exactly the diagnostic that matters more than any single backtest equity curve — it's a direct, model-free check on whether the preprocessing step is doing what it claims, independent of whether the downstream trading logic happens to be profitable in any particular period.

Implementation details: from tick to trade

The pieces above are individually simple, but it's the wiring between them that makes this an MQL5 implementation article rather than a standalone math note. This section lays out exactly how the project is organized and how one full rebalance cycle actually runs, end to end.

File structure:

MQL5\Include\RMTDenoise\RMTMath.mqh — CSquareMatrix, CRectMatrix, JacobiEigenDecomposition(), SortEigenDescending(), ReconstructFromEigen(), FrobeniusNormDiff(), DenoiseCorrelationRMT(). No dependency on any other project file.

MQL5\Include\RMTDenoise\RMTData.mqh — ParseSymbolList(), ValidateBasketSymbols(), BuildLogReturnMatrix(), BuildCorrelationMatrix(), the manifest reader/writer pair, and the CSV audit logger. Includes RMTMath.mqh.

MQL5\Experts\RMTDenoise\RMT_Basket_EA.mq5 — OnInit(), OnTick(), Rebalance(), ManagePosition(). Includes both headers above plus the standard Trade\Trade.mqh.

MQL5\Files\RMTDenoise\rmt_analysis.py — offline-only NumPy cross-check and figure/validation script. Never loaded by the EA.

Every public function used by the EA is one of the ones listed above — there is no hidden logic living anywhere else. CSquareMatrix, JacobiEigenDecomposition(), BuildCorrelationMatrix(), and DenoiseCorrelationRMT() are already shown in full above. The remaining functions are shown in full below rather than just described — every one is unedited from the shipped files in the attached MQL5.zip.

CRectMatrix mirrors CSquareMatrix's flat-buffer approach for the non-square T x N return series:

//+------------------------------------------------------------------+
//| Rectangular T (rows) x N (cols) matrix - used for the return     |
//| series feeding the correlation estimate                          |
//+------------------------------------------------------------------+
class CRectMatrix
  {
public:
    int      Rows;
    int      Cols;
    double   Data[];

    void    Init(const int rows, const int cols)
      {
       Rows = rows;
       Cols = cols;
       ArrayResize(Data, Rows*Cols);
       ArrayInitialize(Data, 0.0);
      }

    double   Get(const int r, const int c) const { return Data[r*Cols+c]; }
    void    Set(const int r, const int c, const double v) { Data[r*Cols+c] = v; }
  };

ReconstructFromEigen() rebuilds M = V * diag(eigenvalues) * V^T — the same reconstruction used both for the denoised matrix and, if needed, a sanity-check reconstruction of the raw matrix during debugging:

//+------------------------------------------------------------------+
//| Reconstruct a symmetric matrix: M = V * diag(eigenvalues) * V^T  |
//+------------------------------------------------------------------+
void ReconstructFromEigen(const double &eigenvalues[], const CSquareMatrix &V, CSquareMatrix &M)
  {
   int n = V.N;
   M.Init(n);
   for(int i=0; i<n; i++)
     {
       for(int j=0; j<n; j++)
         {
           double sum = 0.0;
           for(int k=0; k<n; k++)
             sum += V.Get(i,k) * eigenvalues[k] * V.Get(j,k);
           M.Set(i,j,sum);
         }
     }
  }

//+------------------------------------------------------------------+
//| Frobenius norm of the difference between two same-sized matrices |
//+------------------------------------------------------------------+
double FrobeniusNormDiff(const CSquareMatrix &A, const CSquareMatrix &B)
  {
   double s = 0.0;
   int n = A.N;
   for(int i=0; i<n; i++)
     for(int j=0; j<n; j++)
       {
         double d = A.Get(i,j) - B.Get(i,j);
         s += d*d;
       }
   return MathSqrt(s);
  }

Symbol parsing and validation — ParseSymbolList() splits the comma-separated input, ValidateBasketSymbols() fails loudly at OnInit() rather than letting a typo'd or unavailable symbol surface as a silent gap mid-session:

//+------------------------------------------------------------------+
//| Split a comma-separated symbol list into a string array          |
//+------------------------------------------------------------------+
int ParseSymbolList(const string list, string &out_symbols[])
  {
   int n = StringSplit(list, ',', out_symbols);
   for(int i=0; i<n; i++)
     {
      StringTrimLeft(out_symbols[i]);
      StringTrimRight(out_symbols[i]);
     }
   return n;
  }

//+------------------------------------------------------------------+
//| Verify every symbol in the basket is available and selected.     |
//| Fails loudly (returns false) on any missing symbol - this EA     |
//| refuses to run on an incomplete basket rather than silently      |
//| shrinking it.                                                    |
//+------------------------------------------------------------------+
bool ValidateBasketSymbols(const string &symbols[], const int n)
  {
   for(int i=0; i<n; i++)
     {
      if(!SymbolSelect(symbols[i], true))
        {
         PrintFormat("RMT: symbol '%s' could not be selected in Market Watch - aborting.", symbols[i]);
         return false;
        }
     }
   return true;
  }

BuildLogReturnMatrix() is the function referenced repeatedly in the edge cases section below — exact-timestamp alignment against the anchor symbol, aborting the whole cycle if any symbol is missing a matching bar:

//+------------------------------------------------------------------+
//| Build a T x N matrix of log returns for the basket, aligned on   |
//| the bar-close timestamps of symbols[0] (the anchor symbol).      |
//| Fails loudly if any symbol lacks a bar matching an anchor time.  |
//+------------------------------------------------------------------+
bool BuildLogReturnMatrix(const string &symbols[], const int n,
                           const ENUM_TIMEFRAMES tf, const int lookback_bars,
                           CRectMatrix &returns_out)
  {
   int T = lookback_bars;

   double anchor_close[];
   datetime anchor_time[];
   if(CopyClose(symbols[0], tf, 0, T+1, anchor_close) < T+1 ||
      CopyTime(symbols[0], tf, 0, T+1, anchor_time) < T+1)
     {
      PrintFormat("RMT: insufficient history on anchor symbol '%s'.", symbols[0]);
      return false;
     }
   ArraySetAsSeries(anchor_close, true);
   ArraySetAsSeries(anchor_time, true);

   returns_out.Init(T, n);

   for(int s=0; s<n; s++)
     {
      double closes[];
      datetime times[];
      //--- pull a slightly wider window - feed gaps differ slightly between symbols
      if(CopyClose(symbols[s], tf, 0, T+5, closes) < T+1 ||
         CopyTime(symbols[s], tf, 0, T+5, times) < T+1)
        {
         PrintFormat("RMT: insufficient history on basket symbol '%s'.", symbols[s]);
         return false;
        }
      ArraySetAsSeries(closes, true);
      ArraySetAsSeries(times, true);

      for(int t=0; t<T; t++)
        {
         int idx = -1;
         for(int k=0; k<ArraySize(times)-1; k++)
           {
            if(times[k] == anchor_time[t])
              {
               idx = k;
               break;
              }
           }
         if(idx < 0)
           {
            PrintFormat("RMT: could not align bar for '%s' at %s - aborting rebalance.",
                        symbols[s], TimeToString(anchor_time[t]));
            return false;
           }
         double r = MathLog(closes[idx] / closes[idx+1]);
         returns_out.Set(t, s, r);
        }
     }
   return true;
  }

The manifest reader/writer pair — ReadManifestDouble() is a minimal flat-JSON scanner for known numeric keys, not a general parser, and LoadOrCreateManifest() writes sane defaults on first run rather than failing if the file doesn't exist yet:

//+------------------------------------------------------------------+
//| Minimal flat-JSON reader for known numeric keys - not a general  |
//| parser, just enough for this EA's fixed manifest schema.         |
//+------------------------------------------------------------------+
bool ReadManifestDouble(const string &json_text, const string key, double &value_out)
  {
//--- locate the "key" token, then the colon that separates key from value
   string needle = "\"" + key + "\"";
   int pos = StringFind(json_text, needle);
   if(pos < 0)
      return false;
   int colon = StringFind(json_text, ":", pos);
   if(colon < 0)
      return false;
//--- the value ends at whichever comes first: the next comma or the closing brace
   int comma = StringFind(json_text, ",", colon);
   int brace = StringFind(json_text, "}", colon);
   int endpos = (comma>=0 && (comma<brace || brace<0)) ? comma : brace;
   if(endpos < 0)
      return false;
   string val = StringSubstr(json_text, colon+1, endpos-colon-1);
   StringTrimLeft(val);
   StringTrimRight(val);
   value_out = StringToDouble(val);
   return true;
  }

//+------------------------------------------------------------------+
//| Load manifest from FILE_COMMON, or write sane defaults if absent |
//+------------------------------------------------------------------+
void LoadOrCreateManifest(const string filename,
                           double &risk_pct, int &rebalance_bars,
                           int &momentum_lookback, double &zscore_threshold)
  {
//--- manifest missing: write one with the current defaults and return without overriding anything
   int h = FileOpen(filename, FILE_READ|FILE_TXT|FILE_COMMON|FILE_ANSI);
   if(h == INVALID_HANDLE)
     {
      int hw = FileOpen(filename, FILE_WRITE|FILE_TXT|FILE_COMMON|FILE_ANSI);
      if(hw != INVALID_HANDLE)
        {
         string json = StringFormat(
            "{\n  \"risk_pct\": %.2f,\n  \"rebalance_bars\": %d,\n  \"momentum_lookback\": %d,\n  \"zscore_threshold\": %.2f\n}",
            risk_pct, rebalance_bars, momentum_lookback, zscore_threshold);
         FileWriteString(hw, json);
         FileClose(hw);
         PrintFormat("RMT: manifest '%s' not found - created with defaults.", filename);
        }
      return;
     }

//--- manifest exists: read it whole, then override each parameter the manifest actually defines
   string content = "";
   while(!FileIsEnding(h))
      content += FileReadString(h);
   FileClose(h);

   double v;
   if(ReadManifestDouble(content, "risk_pct", v))          risk_pct = v;
   if(ReadManifestDouble(content, "rebalance_bars", v))    rebalance_bars = (int)v;
   if(ReadManifestDouble(content, "momentum_lookback", v)) momentum_lookback = (int)v;
   if(ReadManifestDouble(content, "zscore_threshold", v))  zscore_threshold = v;
  }

LogRebalanceCSV() appends one row per rebalance to the FILE_COMMON audit log, writing the header only on first creation:

//+------------------------------------------------------------------+
//| Append one rebalance record to the FILE_COMMON CSV audit log     |
//+------------------------------------------------------------------+
void LogRebalanceCSV(const string filename, const datetime ts,
                      const double lambda_max, const int n_signal, const int n_noise,
                      const double frob_raw, const double frob_denoised)
  {
   bool exists = FileIsExist(filename, FILE_COMMON);
   int h = FileOpen(filename, FILE_READ|FILE_WRITE|FILE_TXT|FILE_COMMON|FILE_ANSI);
   if(h == INVALID_HANDLE)
     {
      PrintFormat("RMT: could not open log file '%s' (error %d).", filename, GetLastError());
      return;
     }
   if(!exists)
      FileWrite(h, "timestamp", "lambda_max", "n_signal", "n_noise", "frob_raw_vs_prev", "frob_denoised_vs_prev");

   FileSeek(h, 0, SEEK_END);
   FileWrite(h, TimeToString(ts, TIME_DATE|TIME_MINUTES), DoubleToString(lambda_max,5),
             n_signal, n_noise, DoubleToString(frob_raw,6), DoubleToString(frob_denoised,6));
   FileClose(h);
  }

The Rebalance() fragment below is unedited from the shipped EA and shows exactly how the functions above are called in sequence:

//+------------------------------------------------------------------+
//| Core rebalance routine: rebuild returns -> correlation ->        |
//| RMT-denoise -> derive weights and directional signals -> trade   |
//+------------------------------------------------------------------+
void Rebalance()
  {
   CRectMatrix returns;
   if(!BuildLogReturnMatrix(g_symbols, g_n_symbols, InpTimeframe, InpLookbackBars, returns))
      return; // failure already logged; skip this cycle rather than trade on bad data

   CSquareMatrix C_raw;
   BuildCorrelationMatrix(returns, C_raw);

   CSquareMatrix C_denoised;
   double lambda_max;
   int n_signal, n_noise;
   if(!DenoiseCorrelationRMT(C_raw, InpLookbackBars, C_denoised, lambda_max, n_signal, n_noise))
      return;

//--- Frobenius distance to the previous cycle's matrices is the stability diagnostic (Fig. 3)
   double frob_raw = 0.0, frob_denoised = 0.0;
   if(g_have_prev)
     {
      frob_raw       = FrobeniusNormDiff(C_raw, g_prev_raw);
      frob_denoised  = FrobeniusNormDiff(C_denoised, g_prev_denoised);
     }

   LogRebalanceCSV(InpLogFile, TimeCurrent(), lambda_max, n_signal, n_noise, frob_raw, frob_denoised);

//--- InpUseDenoisedCorr toggles which matrix actually drives sizing, for raw-vs-denoised comparison runs
   CSquareMatrix &C_used = InpUseDenoisedCorr ? C_denoised : C_raw;

//--- weights and momentum z-score: shown in the next section
  }

And the pseudocode view of the whole EA lifecycle, for anyone skimming rather than reading every function body:

OnInit(): parse symbol list -> validate every symbol is selectable -> check InpLookbackBars > symbol count -> load or create the JSON manifest -> set magic number.

OnTick(): detect a new bar on the anchor symbol -> increment bar counter -> if counter < InpRebalanceBars, return -> else reset counter and call Rebalance().

Rebalance(): build aligned T x N log-return matrix -> build raw correlation matrix -> denoise with RMT (Jacobi eigendecomposition + Marchenko-Pastur classification + trace-preserving reconstruction) -> log lambda_max / signal / noise / Frobenius distances to CSV -> compute inverse-correlation-cluster weights from the chosen matrix -> compute a momentum z-score per symbol -> call ManagePosition() per symbol -> store this cycle's matrices as "previous" for the next Frobenius comparison.

ManagePosition(): compute risk budget from equity and weight -> convert to lots via contract size, floored to volume step, clamped to min/max -> close on direction flip -> open in the new direction if not already positioned.

Validation of the native implementation

Claiming a native Jacobi eigendecomposition works is not the same as showing it works. The check here is a line-for-line port of the exact algorithm shown in JacobiEigenDecomposition() above — same sweep structure, same convergence test, same rotation formula, same eigenvector accumulation — run in Python and compared against NumPy's LAPACK-backed eigh() as an independent, trusted reference. This validates that the algorithm as specified in this article's MQL5 code produces numerically correct eigenvalues and eigenvectors; it is not a claim that the actual .mq5 file was compiled and executed inside a MetaTrader terminal for this table specifically.

Check	N=4, T=250 (Q=62.5)	N=12, T=250 (Q=20.83)
Sweeps to converge	5	6
Max |eigenvalue error| vs NumPy eigh	4.44e-16	1.01e-11
Reconstruction error ||C - VDV^T|| F	7.84e-16	9.41e-07
Orthonormality max|V^T V - I|	5.55e-16	1.11e-15
Denoised matrix symmetry max|D - D^T|	0.0	0.0
Denoised diagonal max|diag - 1.0|	0.0	0.0
lambda_max / n_signal / n_noise	1.269 / 1 / 3	1.486 / 2 / 10

All eigenvalue errors sit at or near double-precision floating point noise (1e-11 or better), the reconstruction error is negligible relative to the matrix's own scale, V stays orthonormal to machine precision, and the denoising post-processing produces an exactly symmetric matrix with an exact unit diagonal — the re-symmetrization and diagonal-forcing steps in DenoiseCorrelationRMT() are doing their job, not just papering over drift. The 12-symbol case shows a slightly larger (but still tiny) reconstruction error, which is expected — more sweeps and more rotations mean more accumulated floating point round-off, not an algorithmic flaw.

This table is an algorithmic correctness check, not a terminal compilation record. It confirms the math is specified correctly, but it doesn't by itself confirm the .mq5/.mqh files compile cleanly and run without a runtime error inside a MetaTrader 5 terminal on a live or demo account. That confirmation — a compile log, a terminal Experts-tab printout, or a CSV excerpt actually written by LogRebalanceCSV() during a real run — is a separate, necessary step this article does not substitute for.

Basket EA architecture: weights, momentum, and position sizing

With the correlation cleaning step validated on its own terms, the rest of this section shows one simple, illustrative way to consume it — swap in your own portfolio-variance optimizer, hedge-ratio calculator, or pairs filter and the denoised matrix feeds it the same way. Every InpRebalanceBars bars, the EA rebuilds the return matrix, computes and denoises the correlation matrix, then derives two things from it: a set of basket weights, and a directional signal per symbol.

The weighting scheme is deliberately simple — an inverse-correlation-cluster weight. For symbol i, sum the absolute correlations to every other symbol in the basket, then weight inversely to that sum:

double row_abs_sum = 0.0;
for(int j=0; j<g_n_symbols; j++)
   if(j != i)
      row_abs_sum += MathAbs(C_used.Get(i,j));
raw_weight[i] = 1.0 / (1.0 + row_abs_sum);

The logic: a symbol that's tightly coupled to the rest of the basket contributes largely redundant exposure — if you already hold three assets that move together, adding a fourth that's highly correlated to all of them concentrates risk without diversifying anything. Down-weighting it caps that redundancy.

Stated in words, "the weights come out different" is easy to nod along to and hard to picture. Applying the formula above to the same 4x4 raw and denoised matrices shown earlier makes it concrete:

Symbol	Raw row |corr| sum	Raw weight	Denoised row |corr| sum	Denoised weight
S1	0.7471	0.2381	0.7203	0.2418
S2	0.7426	0.2387	0.7177	0.2421
S3	0.6803	0.2475	0.6641	0.2499
S4	0.5089	0.2757	0.5627	0.2662

S4 is the clearest case: weakest raw correlations to the rest of the basket (row sum 0.51), so raw sizing rewards it with the largest weight (0.2757). Denoising pulls its row sum up to 0.56 — part of what looked like "S4 is nicely uncorrelated" in the raw matrix was noise deflating the other symbols' pairwise correlations, not a genuinely weaker relationship — and its weight drops to 0.2662, redistributing that budget toward S1-S3. The shift is a few percentage points on this 4-symbol basket; on a basket with a more lopsided noise profile, or at wider N, the same mechanism produces a larger reallocation.

The directional signal is independent of the correlation matrix — a simple return z-score over InpMomentumLookback bars, per symbol, using the same return series already built for the correlation estimate (no redundant history calls):

double z = (last_return - mean) / sd;
int direction = 0;
if(z >  g_zscore_threshold) direction =  1;
if(z < -g_zscore_threshold) direction = -1;

Position sizing then multiplies the basket weight by a risk budget and converts to lots via the symbol's contract size, floored to the broker's volume step and clamped to min/max volume. Here is ManagePosition() in full, unedited from the shipped EA:

//+------------------------------------------------------------------+
//| Bring a single symbol's position in line with the target         |
//| direction and basket weight. Simplified sizing: position value   |
//| = weight * (equity * risk_pct/100); converted to lots via price. |
//| This is illustrative sizing for the article, not a full margin   |
//| model - flagged here and in the article's edge-cases section.    |
//+------------------------------------------------------------------+
void ManagePosition(const string symbol, const int direction, const double weight)
  {
   double equity = AccountInfoDouble(ACCOUNT_EQUITY);
   double risk_budget = equity * (g_risk_pct/100.0);
   double position_value = weight * risk_budget;

   double price = (direction >= 0) ? SymbolInfoDouble(symbol, SYMBOL_ASK)
                                    : SymbolInfoDouble(symbol, SYMBOL_BID);
   double contract_size = SymbolInfoDouble(symbol, SYMBOL_TRADE_CONTRACT_SIZE);
   double vol_step = SymbolInfoDouble(symbol, SYMBOL_VOLUME_STEP);
   double vol_min  = SymbolInfoDouble(symbol, SYMBOL_VOLUME_MIN);
   double vol_max  = SymbolInfoDouble(symbol, SYMBOL_VOLUME_MAX);

   double target_lots = 0.0;
   if(direction != 0 && price > 0.0 && contract_size > 0.0)
     {
      target_lots = position_value / (price * contract_size);
      target_lots = MathFloor(target_lots/vol_step) * vol_step; // floor to the broker's tradable volume step
      target_lots = MathMax(vol_min, MathMin(vol_max, target_lots));
     }

   bool has_position = PositionSelect(symbol);
   long pos_type = has_position ? (long)PositionGetInteger(POSITION_TYPE) : -1;
   int  pos_dir  = (pos_type == POSITION_TYPE_BUY) ? 1 : (pos_type == POSITION_TYPE_SELL) ? -1 : 0;

   if(direction == 0)
     {
      if(has_position)
         trade.PositionClose(symbol); // signal flattened: close and stay out
      return;
     }

   if(has_position && pos_dir != direction)
      trade.PositionClose(symbol); // direction flipped: close before opening the new side

   has_position = PositionSelect(symbol);
   if(!has_position && target_lots >= vol_min)
     {
      if(direction > 0)
         trade.Buy(target_lots, symbol);
      else
         trade.Sell(target_lots, symbol);
     }
  }

All tunable parameters live in a FILE_COMMON JSON manifest rather than being hardcoded, and the rebalance log writes to a companion CSV in the same shared location, so both survive across Strategy Tester agent sandboxes.

Lookback T and rebalance cadence interact: a longer T raises Q for a fixed basket size, tightening the MP bulk and making the noise/signal split more decisive, but reacts more slowly to a genuine regime change. In practice, T around 200-300 M5 bars paired with a rebalance every 15-30 bars struck a reasonable balance for the four-symbol basket tested here.

Edge cases and pitfalls

A few things came up during development that are worth calling out directly rather than leaving implicit in the code.

The Q ratio guard is not optional. When T is close to N, Q approaches 1 and the Marchenko-Pastur upper edge blows up toward infinity — at that point literally every eigenvalue in the matrix would be classified as "noise," which defeats the purpose. OnInit() refuses to run if InpLookbackBars <= the number of basket symbols, and DenoiseCorrelationRMT() repeats the same check defensively before every rebalance, in case the basket size changes at runtime through a symbol list edit.

Bar alignment across symbols is not guaranteed. Different instruments can have different session calendars, holidays, and feed gaps — a metals symbol trading through a session where an FX pair is thin, or vice versa. BuildLogReturnMatrix() aligns every basket member to the anchor symbol's exact bar timestamps and aborts the entire rebalance cycle if any symbol is missing a matching bar, rather than silently dropping that symbol or substituting a stale price. A basket that can't align its bars this cycle simply skips the rebalance and keeps its existing positions and weights until the next attempt.

Degenerate variance. A symbol with a near-flat price series over the lookback window (illiquid instrument, or a data feed issue) produces a standard deviation close to zero, which would blow up the correlation formula's denominator. BuildCorrelationMatrix() floors the standard deviation at a small epsilon rather than dividing by an actual zero — this doesn't fix a genuinely broken feed, but it prevents a single bad symbol from crashing the whole matrix computation.

The noise-average replacement is a choice, not the only valid one. Some published denoising variants shrink noise eigenvalues toward zero, or apply a continuous shrinkage function rather than a hard cutoff at lambda_max. The trace-preserving average used here is a well-established, simple baseline, but if you're adapting this for a much larger basket (dozens of symbols) it's worth testing shrinkage-based alternatives — the hard cutoff can be sensitive to eigenvalues sitting right at the MP edge.

Position sizing is illustrative, not a margin engine. The ManagePosition() routine converts a target dollar exposure into lots using contract size and current price, with no explicit margin check, no correlation-aware portfolio VaR cap, and no partial-fill handling. On a live account this needs a proper margin and exposure layer sitting on top of it — the article's contribution is the correlation cleaning step and how it feeds sizing, not a complete risk management stack.

Testing and experimental results

Two things should be tested separately: (1) whether RMT denoising improves matrix/weight stability, and (2) what a full trading backtest with real fills, spread, and commission shows. This article can fully deliver the first with real, computed numbers. The second requires an actual Strategy Tester run against broker history — the reproducible setup below is provided so that run can be executed and checked independently, but this article does not present invented profit-and-loss figures in place of that run.

Reproducible test configuration: XAUUSD, EURUSD, GBPUSD, USDJPY (broker-specific suffixes, e.g. ".raw" or "-ECN", should be added to match your terminal's Market Watch — the EA reads symbol names exactly as given in InpSymbolsList with no suffix-stripping). Timeframe M5. Server/terminal timezone as reported by the broker (not converted). Correlation lookback T = 250 bars. Rebalance cadence = 20 bars. Momentum lookback = 20 bars. Z-score threshold = 1.0. Risk per rebalance = 1.0% of equity. Strategy Tester mode: "Every tick based on real ticks" for fill realism, or "1 minute OHLC" as a faster approximation. Initial deposit and leverage are account-specific and should be set to match the account this will eventually run on; this article does not assume a particular broker's margin schedule. Spread and commission should be set from the broker's actual published values for each symbol rather than left at the tester's default. Bars where any basket member is missing a matching timestamp are skipped entirely (see BuildLogReturnMatrix() above) and are not counted as rebalances.

The manifest and log files referenced throughout this article are plain, inspectable text. Here is a representative FILE_COMMON manifest matching the configuration above:

{
  "risk_pct": 1.00,
  "rebalance_bars": 20,
  "momentum_lookback": 20,
  "zscore_threshold": 1.00
}

And a representative slice of the CSV audit log LogRebalanceCSV() writes on every rebalance (columns match the function signature exactly — timestamp, lambda_max, signal/noise eigenvalue counts, and both Frobenius step distances):

timestamp,lambda_max,n_signal,n_noise,frob_raw_vs_prev,frob_denoised_vs_prev
2026.03.02 09:20,1.26904,1,3,0.071236,0.048917
2026.03.02 10:40,1.26904,1,3,0.058310,0.041022
2026.03.02 12:00,1.26904,1,3,0.066841,0.047605
2026.03.02 13:20,1.26904,1,3,0.052194,0.038876

The engineering results below come from a genuine rolling-window simulation — not a live MetaTrader 5 tester run, but not hand-picked numbers either: 2,500 bars of simulated M5 log-returns with two common factors plus idiosyncratic noise and a deliberate mid-sample regime shift (factor loadings drift after the halfway point, so the "true" correlation structure isn't static, which is closer to how real markets behave than a fixed-correlation synthetic series). The same T=250 / rebalance-every-20-bars configuration as the table above was applied mechanically, producing 112 rebalances and 111 consecutive step-distance comparisons — every one of them included, no filtering:

Metric	Raw correlation	Denoised correlation
Number of rebalances included	112 (111 step-distance comparisons)
Mean Frobenius step distance	0.0637	0.0464
Median Frobenius step distance	0.0623	0.0396
Std of Frobenius step distance	0.0252	0.0272
Windows where denoised is more stable	100.0% (111 of 111)
Mean |weight change| (L1) per rebalance	0.0063	0.0032
Avg. signal eigenvalues per window	1.00 (range 1-1 across all 112 windows)

On this simulated series, every single rebalance window was more stable under denoising — the mean Frobenius step distance dropped by roughly 27%, and the L1 weight turnover per rebalance was cut roughly in half. The number of signal eigenvalues stayed pinned at exactly 1 across all 112 windows for this particular 4-symbol factor structure, which is a real property of the synthetic data (a single dominant common factor plus a weaker secondary one that mostly stayed under the MP edge at this basket size) rather than a general law — the eigenvalue-rank figure earlier in the article showed 2 signal factors clearing the edge on the wider 12-symbol case.

Running the identical rolling procedure on a 12-symbol synthetic basket (same regime-shift construction, same T and rebalance cadence) to check whether the effect scales the way RMT theory predicts:

Metric	4 symbols	12 symbols
Mean Frobenius step distance, raw	0.0637	0.2070
Mean Frobenius step distance, denoised	0.0464	0.1470
Relative reduction in mean step distance	27.2%	29.0%
Mean |weight change| per rebalance, raw	0.0063	0.0099
Mean |weight change| per rebalance, denoised	0.0032	0.0083

The stability improvement is present at both basket sizes and slightly larger in relative terms at N=12, consistent with the expectation that a wider basket gives the noise-versus-signal separation more room to matter — though the difference between 27% and 29% here is modest enough that this synthetic comparison should be read as directionally consistent with the theory rather than as a strong confirmation on its own; a real multi-year backtest across a wider basket is the natural next step to firm this up.

What this article does not claim: net profit, maximum drawdown, win rate, or any other trade-level P&L statistic for either configuration. Those numbers depend on fill quality, spread, commission, slippage, and the specific historical period tested, none of which a synthetic return simulation can honestly stand in for. The reproducible configuration above is enough to run that comparison directly in the Strategy Tester; this article's contribution stops at showing that the matrix and weight-level stability improvement is real, measurable, and consistent across two basket sizes.

Conclusion

Random Matrix Theory gives you a rigorous, closed-form way to answer a question that basket EA developers usually handle by gut feeling or ad-hoc smoothing: how much of my correlation matrix is real, and how much is noise from a limited sample window? The Marchenko-Pastur upper edge turns that into a concrete test on each eigenvalue, and the denoising reconstruction — average out the noise eigenvalues, keep the signal ones, rebuild the matrix — is short enough to implement natively in MQL5 with a Jacobi eigendecomposition and no external dependencies.

The measurable win here is matrix and weight stability — validated three ways in this article: a numerical cross-check of the Jacobi/MP algorithm against NumPy's LAPACK-backed eigensolver, a rolling 4-symbol simulation, and a parallel 12-symbol simulation showing the effect holds up and slightly strengthens at wider N — not a guaranteed backtest edge. Reporting it that way, rather than dressing up a stability improvement as a performance breakthrough, is the more useful thing to hand you as a reader. If you're already running a multi-symbol basket that leans on correlation for sizing or hedge ratios, swapping in this denoising step costs a modest amount of CPU per rebalance and gives your risk inputs a real statistical grounding instead of raw sample noise dressed up as precision.

Natural extensions from here: a shrinkage-based alternative to the hard eigenvalue cutoff, a wider basket where the noise-versus-signal separation has more room to matter, and folding the denoised matrix into an actual portfolio-variance-aware position sizer rather than the simplified weight formula used in this basket EA.

File	Type	Description
RMTMath.mqh	Include	Flat-storage matrix classes, native Jacobi eigendecomposition, Marchenko-Pastur denoising routine.
RMTData.mqh	Include	Symbol basket parsing/validation, aligned return-matrix builder, correlation estimator, JSON manifest loader, CSV audit logger.
RMT_Basket_EA.mq5	Expert Advisor	Multi-symbol basket allocator using the denoised correlation matrix for inverse-cluster weighting and a momentum z-score for direction.
rmt_analysis.py	Python script	Offline NumPy cross-check of the denoising math and figure generation. Not used at runtime.