Discussing the article: "Overcoming The Limitation of Machine Learning (Part 8): Nonparametric Strategy Selection"

 

Check out the new article: Overcoming The Limitation of Machine Learning (Part 8): Nonparametric Strategy Selection.

This article shows how to configure a black-box model to automatically uncover strong trading strategies using a data-driven approach. By using Mutual Information to prioritize the most learnable signals, we can build smarter and more adaptive models that outperform conventional methods. Readers will also learn to avoid common pitfalls like overreliance on surface-level metrics, and instead develop strategies rooted in meaningful statistical insight.

In our previous discussion on automatic strategy selection, we explored two approaches to identifying trading strategies from a list of candidates. The first was a white-box method using matrix factorization—simple, transparent, and intuitive. Today, we turn our attention to performing better at the second approach: the more complex black-box solution.

The challenge of identifying profitable strategies remains significant. This article focuses on improving how black-box models are configured and set up. Previously, we designed a statistical model that learned to predict each strategy’s expected profit, guiding us toward potentially profitable strategies. While this is a valid goal, a simpler alternative would be to identify the strategy our black-box model can learn most effectively—choosing the target it performs “best” on. But this introduces a significant challenge.

Comparing model performance across different regression targets is not straightforward. Unlike classification tasks—where metrics like accuracy or precision make comparisons easy—regression deals with real-valued targets like future returns, and common metrics such as RMSE can mislead. The challenge is that common Euclidean dispersion metrics, are sensitive to scale, meaning indicators like Stochastic and Moving Average values are not directly comparable. In addition to this problem, classical supervised learning offers little guidance here.

Author: Gamuchirai Zororo Ndawana