# Assessing high-order effects in feature importance via predictability decomposition

arXiv: https://arxiv.org/abs/2412.09964 · HTML: https://mlsanity.com/paper/2412.09964/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 13 references that are in the map.
- First posted on arXiv 13 Dec 2024.

## Builds on (most cited first, 3 of 3)

- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [The physics of higher-order interactions in complex systems](https://mlsanity.com/paper/2110.06023/) (2021)
- [Disentangling Interactions and Dependencies in Feature Attribution](https://mlsanity.com/paper/2410.23772/) (2024)

## Built on by (most cited first, 2 of 2)

- [The Representational Limit of Scalar Interactions: An Interventional Decomposition](https://mlsanity.com/paper/2606.19410/) (2026)
- [Cooperative effects in feature importance of individual patterns: application to air pollutants and Alzheimer disease](https://mlsanity.com/paper/2508.00930/) (2025)

## Abstract (arXiv)

Leveraging the large body of work devoted in recent years to describe redundancy and synergy in multivariate interactions among random variables, we propose a novel approach to quantify cooperative effects in feature importance, one of the most used techniques for explainable artificial intelligence. In particular, we propose an adaptive version of a well-known metric of feature importance, named Leave One Covariate Out (LOCO), to disentangle high-order effects involving a given input feature in regression problems. LOCO is the reduction of the prediction error when the feature under consideration is added to the set of all the features used for regression. Instead of calculating the LOCO using all the features at hand, as in its standard version, our method searches for the multiplet of features that maximize LOCO and for the one that minimize it. This provides a decomposition of the LOCO as the sum of a two-body component and higher-order components (redundant and synergistic), also highlighting the features that contribute to building these high-order effects alongside the driving feature. We report the application to proton/pion discrimination from simulated detector measures by GEANT.