# Interpretability via Model Extraction

arXiv: https://arxiv.org/abs/1706.09773 · HTML: https://mlsanity.com/paper/1706.09773/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jun 2017.

## Built on by (most cited first, 30 of 40)

- [The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01602/) (2022)
- [OpenXAI: Towards a Transparent Evaluation of Model Explanations](https://mlsanity.com/paper/2206.11104/) (2022)
- [Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications](https://mlsanity.com/paper/2103.04244/) (2021)
- [Neural Basis Models for Interpretability](https://mlsanity.com/paper/2205.14120/) (2022)
- [Rethinking Explainability as a Dialogue: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01875/) (2022)
- [Logic-Based Explainability in Machine Learning](https://mlsanity.com/paper/2211.00541/) (2022)
- [Towards the Unification and Robustness of Perturbation and Gradient Based Explanations](https://mlsanity.com/paper/2102.10618/) (2021)
- [Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations](https://mlsanity.com/paper/2205.07277/) (2022)
- [Provably efficient, succinct, and precise explanations](https://mlsanity.com/paper/2111.01576/) (2021)
- [M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis](https://mlsanity.com/paper/2107.08264/) (2021)
- [The Role of Explainability in Assuring Safety of Machine Learning in Healthcare](https://mlsanity.com/paper/2109.00520/) (2021)
- [Concept Learning for Interpretable Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2302.12232/) (2023)
- [Dual Student Networks for Data-Free Model Stealing](https://mlsanity.com/paper/2309.10058/) (2023)
- [Towards a theory of model distillation](https://mlsanity.com/paper/2403.09053/) (2024)
- [From Black Boxes to Conversations: Incorporating XAI in a Conversational Agent](https://mlsanity.com/paper/2209.02552/) (2022)
- [A Human-Centric Take on Model Monitoring](https://mlsanity.com/paper/2206.02868/) (2022)
- [Superpolynomial Lower Bounds for Decision Tree Learning and Testing](https://mlsanity.com/paper/2210.06375/) (2022)
- [A Query-Optimal Algorithm for Finding Counterfactuals](https://mlsanity.com/paper/2207.07072/) (2022)
- [A Theory of Interpretable Approximations](https://mlsanity.com/paper/2406.10529/) (2024)
- [Recent Advances in Malware Detection: Graph Learning and Explainability](https://mlsanity.com/paper/2502.10556/) (2025)
- [Model Doctor: A Simple Gradient Aggregation Strategy for Diagnosing and Treating CNN Classifiers](https://mlsanity.com/paper/2112.04934/) (2021)
- [Trade-off Between Efficiency and Consistency for Removal-based Explanations](https://mlsanity.com/paper/2210.17426/) (2022)
- [Implementing Responsible AI: Tensions and Trade-Offs Between Ethics Aspects](https://mlsanity.com/paper/2304.08275/) (2023)
- [Properly Learning Decision Trees with Queries Is NP-Hard](https://mlsanity.com/paper/2307.04093/) (2023)
- [Model-Agnostic Policy Explanations with Large Language Models](https://mlsanity.com/paper/2504.05625/) (2025)
- [Explaining Agent Behavior with Large Language Models](https://mlsanity.com/paper/2309.10346/) (2023)
- [Understanding Your Agent: Leveraging Large Language Models for Behavior Explanation](https://mlsanity.com/paper/2311.18062/) (2023)
- [i-Align: an interpretable knowledge graph alignment model](https://mlsanity.com/paper/2308.13755/) (2023)
- [Debunking Grad-ECLIP: A Comprehensive Study on Its Incorrectness and Fundamental Principles for Model Interpretation](https://mlsanity.com/paper/2605.12952/) (2026)
- [BODAME: Bilevel Optimization for Defense Against Model Extraction](https://mlsanity.com/paper/2103.06797/) (2021)

## Abstract (arXiv)

The ability to interpret machine learning models has become increasingly important now that machine learning is used to inform consequential decisions. We propose an approach called model extraction for interpreting complex, blackbox models. Our approach approximates the complex model using a much more interpretable model; as long as the approximation quality is good, then statistical properties of the complex model are reflected in the interpretable model. We show how model extraction can be used to understand and debug random forests and neural nets trained on several datasets from the UCI Machine Learning Repository, as well as control policies learned for several classical reinforcement learning problems.