# Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation

arXiv: https://arxiv.org/abs/1710.06169 · HTML: https://mlsanity.com/paper/1710.06169/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2017.

## Built on by (most cited first, 30 of 62)

- [Does Knowledge Distillation Really Work?](https://mlsanity.com/paper/2106.05945/) (2021)
- [Rethinking Interpretability in the Era of Large Language Models](https://mlsanity.com/paper/2402.01761/) (2024)
- [NODE-GAM: Neural Generalized Additive Model for Interpretable Deep Learning](https://mlsanity.com/paper/2106.01613/) (2021)
- [Explaining black box text modules in natural language with language models](https://mlsanity.com/paper/2305.09863/) (2023)
- [How can I choose an explainer? An Application-grounded Evaluation of Post-hoc Explanations](https://mlsanity.com/paper/2101.08758/) (2021)
- [Rethinking Explainability as a Dialogue: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01875/) (2022)
- [The Who in XAI: How AI Background Shapes Perceptions of AI Explanations](https://mlsanity.com/paper/2107.13509/) (2021)
- [Learning from Teaching Regularization: Generalizable Correlations Should be Easy to Imitate](https://mlsanity.com/paper/2402.02769/) (2024)
- [Survey: Leakage and Privacy at Inference Time](https://mlsanity.com/paper/2107.01614/) (2021)
- [The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanations](https://mlsanity.com/paper/2205.03295/) (2022)
- [Denoising diffusion algorithm for inverse design of microstructures with fine-tuned nonlinear material properties](https://mlsanity.com/paper/2302.12881/) (2023)
- [Challenges in Applying Explainability Methods to Improve the Fairness of NLP Models](https://mlsanity.com/paper/2206.03945/) (2022)
- [From Human Explanation to Model Interpretability: A Framework Based on Weight of Evidence](https://mlsanity.com/paper/2104.13299/) (2021)
- [Data-Efficient and Interpretable Tabular Anomaly Detection](https://mlsanity.com/paper/2203.02034/) (2022)
- [Towards Explainable Evaluation Metrics for Natural Language Generation](https://mlsanity.com/paper/2203.11131/) (2022)
- [Towards a theory of model distillation](https://mlsanity.com/paper/2403.09053/) (2024)
- [Being Right for Whose Right Reasons?](https://mlsanity.com/paper/2306.00639/) (2023)
- [Why Should I Choose You? AutoXAI: A Framework for Selecting and Tuning eXplainable AI Solutions](https://mlsanity.com/paper/2210.02795/) (2022)
- [FedDKD: Federated Learning with Decentralized Knowledge Distillation](https://mlsanity.com/paper/2205.00706/) (2022)
- [Higher-order Neural Additive Models: An Interpretable Machine Learning Model with Feature Interactions](https://mlsanity.com/paper/2209.15409/) (2022)
- [FAT: Learning Low-Bitwidth Parametric Representation via Frequency-Aware Transformation](https://mlsanity.com/paper/2102.07444/) (2021)
- [Explainable Diabetic Retinopathy Detection and Retinal Image Generation](https://mlsanity.com/paper/2107.00296/) (2021)
- [GRAPHSHAP: Explaining Identity-Aware Graph Classifiers Through the Language of Motifs](https://mlsanity.com/paper/2202.08815/) (2022)
- [Understanding Disparities in Post Hoc Machine Learning Explanation](https://mlsanity.com/paper/2401.14539/) (2024)
- [Accurate Explanation Model for Image Classifiers using Class Association Embedding](https://mlsanity.com/paper/2406.07961/) (2024)
- [Robust ML Auditing using Prior Knowledge](https://mlsanity.com/paper/2505.04796/) (2025)
- [Users are the North Star for AI Transparency](https://mlsanity.com/paper/2303.05500/) (2023)
- [Patch Shortcuts: Interpretable Proxy Models Efficiently Find Black-Box Vulnerabilities](https://mlsanity.com/paper/2104.11691/) (2021)
- [Monotonicity for AI ethics and society: An empirical study of the monotonic neural additive model in criminology, education, health care, and finance](https://mlsanity.com/paper/2301.07060/) (2023)
- [How to address monotonicity for model risk management?](https://mlsanity.com/paper/2305.00799/) (2023)

## Abstract (arXiv)

Black-box risk scoring models permeate our lives, yet are typically proprietary or opaque. We propose Distill-and-Compare, a model distillation and comparison approach to audit such models. To gain insight into black-box models, we treat them as teachers, training transparent student models to mimic the risk scores assigned by black-box models. We compare the student model trained with distillation to a second un-distilled transparent model trained on ground-truth outcomes, and use differences between the two models to gain insight into the black-box model. Our approach can be applied in a realistic setting, without probing the black-box model API. We demonstrate the approach on four public data sets: COMPAS, Stop-and-Frisk, Chicago Police, and Lending Club. We also propose a statistical test to determine if a data set is missing key features used to train the black-box model. Our test finds that the ProPublica data is likely missing key feature(s) used in COMPAS.