Discussing the article: "Partial Information Decomposition: When Two Indicators Together Say More Than Either Alone"

[MetaQuotes]

Check out the new article: Partial Information Decomposition: When Two Indicators Together Say More Than Either Alone.

We introduce a Partial Information Decomposition library for MQL5 that decomposes two sources about a target into four atoms: unique to each, shared, and synergy. The implementation uses quantile binning, tabulated logarithms, and a maximum-entropy fit (for I_ccs), and it pairs results with a block-permutation null because atoms sit above zero on finite samples. Use it to screen indicator pairs and judge significance, including family-wise correction.

Start from what Shannon information already gives you. For a target T and sources X1 and X2 there are three mutual informations computable directly from the joint distribution: I(T;X1), I(T;X2), and I(T;X1,X2) with the pair treated as one compound variable. In terms of the four atoms they decompose as:

I(T;X1) = U1 + R
I(T;X2) = U2 + R
I(T;X1,X2) = U1 + U2 + R + S

Three equations, four unknowns, and no cleverer algebra closes the gap. Shannon information alone does not fix the decomposition. The standard extra choice is a definition of redundancy: say what R means and the rest follow, since U1 = I(T;X1) - R, U2 = I(T;X2) - R, and S = I(T;X1,X2) - I(T;X1) - I(T;X2) + R.

One consequence is worth pinning down first. Subtracting the third equation from the sum of the first two gives R - S = I(T;X1) + I(T;X2) - I(T;X1,X2), the co-information, which is fixed by the data and holds for any redundancy measure. For any redundancy axiom, R - S must reproduce the co-information computed directly from the data. It is the strongest internal check available, and it holds to 0.000e+00 on the analytic gates.

The canonical way to see what the four atoms mean is to decompose logic gates whose answers are known on paper: RDN (both sources copy the target), UNQ1 (the target is the first source, the second an independent coin), XOR, AND, OR, and COPY (the target is the pair).

Author: Hammad Dilber