mlsanityData to 13 Oct 2026

Ultra-marginal Feature Importance: Learning from Data with Causal Guarantees

arXiv 2204.09938 ↗First posted 21 Apr 2022stat.ML cs.IT cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Scientists frequently prioritize learning from data rather than training the best possible model; however, research in machine learning often prioritizes the latter. Marginal contribution feature importance (MCI) was developed to break this trend by providing a useful framework for quantifying the relationships in data. In this work, we aim to improve upon the theoretical properties, performance, and runtime of MCI by introducing ultra-marginal feature importance (UMFI), which uses dependence removal techniques from the AI fairness literature as its foundation. We first propose axioms for feature importance methods that seek to explain the causal and associative relationships in data, and we prove that UMFI satisfies these axioms under basic assumptions. We then show on real and simulated data that UMFI performs better than MCI, especially in the presence of correlated interactions and unrelated features, while partially learning the structure of the causal graph and reducing the exponential runtime of MCI to super-linear.

Key facts. Cited by 6 papers in the map (data through 13 Oct 2026). Builds on 34 of its 138 references that are in the map. First posted on arXiv 21 Apr 2022.

Builds on

34 of 138 references.

References · 34

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,515
  2. 2020Explaining by Removing: A Unified Framework for Model Explanationcited by 218
  3. 2016Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Modelscited by 212
  4. 2021D'ya like DAGs? A Survey on Structure Learning and Causal Discoverycited by 210
  5. 2015ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and Rcited by 172
  6. 2021Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Gamecited by 159
  7. 2020True to the Model or True to the Data?cited by 124
  8. 2013Stabilitycited by 119
  9. 2018Invariant Representations without Adversarial Trainingcited by 111
  10. 2018Tunability: Importance of Hyperparameters of Machine Learning Algorithmscited by 99
  11. 2019Unrestricted Permutation forces Extrapolation: Variable Importance Requires at least One More Model, or There Is No Free Variable Importancecited by 85
  12. 2018Learning Controllable Fair Representationscited by 83
22 more
  1. 2020Efficient nonparametric statistical inference on population feature importance using Shapley valuescited by 61
  2. 2013Equitability, mutual information, and the maximal information coefficientcited by 45
  3. 2019Purifying Interaction Effects with the Functional ANOVA: An Efficient Algorithm for Recovering Identifiable Additive Modelscited by 41
  4. 2019A simple measure of conditional dependencecited by 38
  5. 2021A Rigorous Information-Theoretic Definition of Redundancy and Relevancy in Feature Selection Based on (Partial) Information Decompositioncited by 25
  6. 2021Relating the Partial Dependence Plot and Permutation Feature Importance to the Data Generating Processcited by 24
  7. 2021SHAFF: Fast and consistent SHApley eFfect estimates via random Forestscited by 22
  8. 2019Learning Fair Representations for Kernel Modelscited by 17
  9. 2019A Novel Approach to the Partial Information Decompositioncited by 16
  10. 2020Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate datacited by 16
  11. 2020Bits and Pieces: Understanding Information Decomposition from Part-whole Relationships and Formal Logiccited by 15
  12. 2020Learning Smooth and Fair Representationscited by 12
  13. 2020Causal Feature Selection via Orthogonal Searchcited by 11
  14. 2020Neural Decomposition: Functional ANOVA with Variational Autoencoderscited by 10
  15. 2020A Weaker Faithfulness Assumption based on Triple Interactionscited by 10
  16. 2021Fair Representations by Compressioncited by 7
  17. 2021Greater than the parts: A review of the information decomposition approach to causal emergencecited by 6
  18. 2017Privacy with Estimation Guaranteescited by 4
  19. 2021From global to local MDI variable importances for random forests and when they are Shapley valuescited by 4
  20. 2020Marginal Contribution Feature Importance -- an Axiomatic Approach for The Natural Casecited by 2
  21. 2020Distance Correlation Sure Independence Screening for Accelerated Feature Selection in Parkinson's Disease Vocal Datacited by 1
  22. 2021Decomposition of Global Feature Importance into Direct and Associative Components (DEDACT)cited by 1

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 4; 2025: 1; 2026: 1.

6 papers.

  1. 2024TERC: A Transfer Entropy Redundancy Criterion for State Variable Selection in Reinforcement Learningcited by 3
  2. 2024Partial Information Decomposition for Data Interpretability and Feature Selectioncited by 3
  3. 2024Feature Selection for Network Intrusion Detectioncited by 3
  4. 2025Stability Selection via Variable Decorrelationcited by 1
  5. 2026Aggregate Models, Not Explanations: Improving Feature Importance Estimationcited by 1
  6. 2024Tackling water table depth modeling via machine learning: From proxy observations to verifiabilitycited by 0