A Benchmark Is Not a Trading Signal: Lessons from ACI's Tests

By Cree Lawson. Research reviewed November 6, 2026.

A dashboard can describe what strategies have experienced without telling you what to trade next. That distinction matters whenever you consider adding a market-regime filter to an Expert Advisor. The Algo Consistency Index (ACI) offers a concrete case study: its published research includes both a limited positive tracking result and negative forecasting and trading-overlay tests.

Ask three separate questions

First, does the reference track the strategies you care about over the same period? Second, does information available now predict their later results? Third, does a specified action based on that information improve outcomes after costs and with comparable risk? Passing the first test does not pass the other two.

ACI's methodology reports a positive trend-family tracking result for one internal live panel, but no corresponding validation for the mean-reversion and session-clock families. In the November 4 external-panel test, all three families failed the stated tracking criteria. That outside panel consists of hypothetical model accounts with simulated fills and imperfect family tags, so it is a different source of evidence with its own limits.

What the negative results change

The forecasting test, T2, did not pass. The T5 study of simple halt and sizing overlays found no family or rule beating its same-window permutation comparison. T6 tested alternative readings and sizing rules without establishing an improvement that survived the specified checks. The project consequently removed the planned sizing multiplier rather than presenting it as a coming feature.

These results do not prove that no conceivable filter could ever work. Some tests had low power, and the examined strategy populations are specific. They do mean that the published evidence does not support using the current reading as a general instruction to increase risk, stop trading or restart an EA.

Do not mistake a standardized gap for money earned

The benchmark-relative card reports alpha20: a standardized account statistic minus a family reference reading, with beta fixed at one. The two inputs use different standardizations. Therefore alpha20 is not a percentage return, cash profit, or measured excess return after controlling for market exposure.

Illustrative arithmetic: an account statistic of 0.4 minus a reference of 0.1 produces a gap of 0.3. That does not mean 30% return, 0.3% return or a 30% chance of a winning trade. Preserve the units and definition when recording the comparison.

Test your proposed filter before relying on it

If you want to investigate a filter independently, specify its rules before looking for the best outcome. Define the signal, decision time, execution delay, position-size limit, restart condition, transaction costs, comparison strategy and evaluation window. Keep a later period untouched during development.

For example, a proposed rule might stop new entries after a reading crosses a fixed threshold. The test must use only readings actually available before each decision. Compare it with the unfiltered EA and with controls that remove a similar amount of trading exposure. Otherwise, a smaller drawdown may simply reflect less time in the market.

If you try many thresholds and report only the winner, include that search in the evaluation. A promising chart selected from dozens of attempts is weaker evidence than a rule specified in advance and confirmed on new observations.

Preserve the history that traders actually saw

There are at least three relevant dates: the market observation, the first publication, and any later revision. ACI's November 5 update published October readings late and revised some earlier values. Its external panel also revises recent rows as members report. The current historical CSV is therefore not, by itself, a record of information available to a trader on each date.

A research journal should preserve the original print, its methodology version and the decision timestamp. If that information is unavailable, label the experiment retrospective rather than implying a live decision test.

A practical weekly review

• Check freshness and sample counts before discussing the reading.
• Separate hypothetical probes, account-panel observations and external model records.
• Confirm that the reference mechanism matches the EA.
• Record operational explanations for unusual results.
• Keep proposed trading rules separate from descriptive observations.
• Revisit failed tests when genuinely new evidence arrives, without rewriting their original outcome.

ACI can organize a disciplined investigation into strategy conditions. Its current research record supports transparency about those conditions and their limitations; it does not establish a shortcut to knowing which EA will make money next.

Research attribution: the ACI methodology and dated validation record, version 0.2, reviewed November 6, 2026. The author is the founder of Prismatic Algos, which publishes ACI. This educational case study reports the project’s own findings, not independent validation. Trading involves risk of loss.