Discussing the article: "MetaTrader 5 Machine Learning Blueprint (Part 21): Feature Importance Analysis"
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Check out the new article: MetaTrader 5 Machine Learning Blueprint (Part 21): Feature Importance Analysis.
Feature importance often understates correlated predictors by spreading one signal across many engineered copies, while unrelated noise can appear higher. We measure this effect against a known ground truth and compare four remedies: permutation importance with purged cross-validation, single-feature models, and clustered impurity versus clustered accuracy. The results include per-method rankings and a per-cluster dilution ratio that help identify true signals and avoid deleting valuable features.
A trained model in this series ends its run by averaging the feature_importances_ of every tree in its ensemble and sorting the result. That column answers one narrow question: across the training set, which columns did the trees split on most productively? It is in-sample, it is specific to tree-based learners, and it has a property that becomes a problem the moment a feature set grows past a handful of columns. Two features carrying the same information are interchangeable at every split, so each is chosen about half the time, and each receives about half the credit.
That is not a rounding error. Engineer one predictor six ways and its six copies can each land below features that carry a fraction of the information, while the ranking looks entirely reasonable. Nothing in the output flags it. The ranking is not wrong about what the trees did; it is wrong about what the features are worth.
López de Prado sets out the diagnosis and three corrections in Advances in Financial Machine Learning (AFML) chapter 8, and a fourth in Machine Learning for Asset Managers (MLAM) section 6.5.2. This article measures the problem on data where the right answer is fixed by construction, then applies each correction and reports which ones recover it. Two of them disagree with each other, and the disagreement is a property of the methods rather than of the data.
Author: Patrick Murimi Njoroge