# Auditing Black-box Models for Indirect Influence

arXiv: https://arxiv.org/abs/1602.07043 · HTML: https://mlsanity.com/paper/1602.07043/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Feb 2016.

## Built on by (most cited first, 30 of 41)

- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [Trustworthy AI: From Principles to Practices](https://mlsanity.com/paper/2110.01167/) (2021)
- [Auditing large language models: a three-layered approach](https://mlsanity.com/paper/2302.08500/) (2023)
- [Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey](https://mlsanity.com/paper/2207.07068/) (2022)
- [Evaluating and Mitigating Discrimination in Language Model Decisions](https://mlsanity.com/paper/2312.03689/) (2023)
- [Trustworthy AI: A Computational Perspective](https://mlsanity.com/paper/2107.06641/) (2021)
- [Explaining Black-Box Algorithms Using Probabilistic Contrastive Counterfactuals](https://mlsanity.com/paper/2103.11972/) (2021)
- [Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning](https://mlsanity.com/paper/2202.05338/) (2022)
- [Removing biased data to improve fairness and accuracy](https://mlsanity.com/paper/2102.03054/) (2021)
- [From Transparency to Accountability and Back: A Discussion of Access and Evidence in AI Auditing](https://mlsanity.com/paper/2410.04772/) (2024)
- [Granger-causal Attentive Mixtures of Experts: Learning Important Features with Neural Networks](https://mlsanity.com/paper/1802.02195/) (2018)
- [Cohort Shapley value for algorithmic fairness](https://mlsanity.com/paper/2105.07168/) (2021)
- [Towards a multi-stakeholder value-based assessment framework for algorithmic systems](https://mlsanity.com/paper/2205.04525/) (2022)
- [Auditing of AI: Legal, Ethical and Technical Approaches](https://mlsanity.com/paper/2407.06235/) (2024)
- [Algorithmic Factors Influencing Bias in Machine Learning](https://mlsanity.com/paper/2104.14014/) (2021)
- [EVNet: An Explainable Deep Network for Dimension Reduction](https://mlsanity.com/paper/2211.15478/) (2022)
- [Feature Responsiveness Scores: Model-Agnostic Explanations for Recourse](https://mlsanity.com/paper/2410.22598/) (2024)
- [From Data to Behavior: Predicting Unintended Model Behaviors Before Training](https://mlsanity.com/paper/2602.04735/) (2026)
- [SLISEMAP: Supervised dimensionality reduction through local explanations](https://mlsanity.com/paper/2201.04455/) (2022)
- [Verifiable Fairness: Privacy-preserving Computation of Fairness for Machine Learning Systems](https://mlsanity.com/paper/2309.06061/) (2023)
- [Threading the Needle of On and Off-Manifold Value Functions for Shapley Explanations](https://mlsanity.com/paper/2202.11919/) (2022)
- [Estimating and Controlling for Equalized Odds via Sensitive Attribute Predictors](https://mlsanity.com/paper/2207.12497/) (2022)
- [FaiREE: Fair Classification with Finite-Sample and Distribution-Free Guarantee](https://mlsanity.com/paper/2211.15072/) (2022)
- [Achieving Fairness in Predictive Process Analytics via Adversarial Learning (Extended Version)](https://mlsanity.com/paper/2410.02618/) (2024)
- [More Data Can Lead Us Astray: Active Data Acquisition in the Presence of Label Bias](https://mlsanity.com/paper/2207.07723/) (2022)
- [Targeted Activation Penalties Help CNNs Ignore Spurious Signals](https://mlsanity.com/paper/2311.12813/) (2023)
- [A Design Framework for operationalizing Trustworthy Artificial Intelligence in Healthcare: Requirements, Tradeoffs and Challenges for its Clinical Adoption](https://mlsanity.com/paper/2504.19179/) (2025)
- [Modeling Implicit Bias with Fuzzy Cognitive Maps](https://mlsanity.com/paper/2112.12713/) (2021)
- [Prediction Sensitivity: Continual Audit of Counterfactual Fairness in Deployed Classifiers](https://mlsanity.com/paper/2202.04504/) (2022)
- [Quantifying Feature Contributions to Overall Disparity Using Information Theory](https://mlsanity.com/paper/2206.08454/) (2022)

## Abstract (arXiv)

Data-trained predictive models see widespread use, but for the most part they are used as black boxes which output a prediction or score. It is therefore hard to acquire a deeper understanding of model behavior, and in particular how different features influence the model prediction. This is important when interpreting the behavior of complex models, or asserting that certain problematic attributes (like race or gender) are not unduly influencing decisions. In this paper, we present a technique for auditing black-box models, which lets us study the extent to which existing models take advantage of particular features in the dataset, without knowing how the models work. Our work focuses on the problem of indirect influence: how some features might indirectly influence outcomes via other, related features. As a result, we can find attribute influences even in cases where, upon further direct examination of the model, the attribute is not referred to by the model at all. Our approach does not require the black-box model to be retrained. This is important if (for example) the model is only accessible via an API, and contrasts our work with other methods that investigate feature influence like feature selection. We present experimental evidence for the effectiveness of our procedure using a variety of publicly available datasets and models. We also validate our procedure using techniques from interpretable learning and feature selection, as well as against other black-box auditing procedures.