# Interpreting Blackbox Models via Model Extraction

arXiv: https://arxiv.org/abs/1705.08504 · HTML: https://mlsanity.com/paper/1705.08504/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2017.

## Built on by (most cited first, 30 of 44)

- [Logic-Based Explainability in Machine Learning](https://mlsanity.com/paper/2211.00541/) (2022)
- [Dual Student Networks for Data-Free Model Stealing](https://mlsanity.com/paper/2309.10058/) (2023)
- [Leveraging Model-based Trees as Interpretable Surrogate Models for Model Distillation](https://mlsanity.com/paper/2310.03112/) (2023)
- [Visual Exploration of Machine Learning Model Behavior with Hierarchical Surrogate Rule Sets](https://mlsanity.com/paper/2201.07724/) (2022)
- [CAIPI in Practice: Towards Explainable Interactive Medical Image Classification](https://mlsanity.com/paper/2204.02661/) (2022)
- [Algorithmic Decision-Making Safeguarded by Human Knowledge](https://mlsanity.com/paper/2211.11028/) (2022)
- [Rationalizing Predictions by Adversarial Information Calibration](https://mlsanity.com/paper/2301.06009/) (2023)
- [Explainable Models via Compression of Tree Ensembles](https://mlsanity.com/paper/2206.07904/) (2022)
- [FDINet: Protecting against DNN Model Extraction via Feature Distortion Index](https://mlsanity.com/paper/2306.11338/) (2023)
- [Revisiting the LiRA Membership Inference Attack Under Realistic Assumptions](https://mlsanity.com/paper/2603.07567/) (2026)
- [Analogies and Feature Attributions for Model Agnostic Explanation of Similarity Learners](https://mlsanity.com/paper/2202.01153/) (2022)
- [Analyzing Machine Learning Models for Credit Scoring with Explainable AI and Optimizing Investment Decisions](https://mlsanity.com/paper/2209.09362/) (2022)
- [Explainable AI using expressive Boolean formulas](https://mlsanity.com/paper/2306.03976/) (2023)
- [Axe the X in XAI: A Plea for Understandable AI](https://mlsanity.com/paper/2403.00315/) (2024)
- [Inference Attacks: A Taxonomy, Survey, and Promising Directions](https://mlsanity.com/paper/2406.02027/) (2024)
- [A survey and taxonomy of methods interpreting random forest models](https://mlsanity.com/paper/2407.12759/) (2024)
- [Understanding with toy surrogate models in machine learning](https://mlsanity.com/paper/2410.05675/) (2024)
- [Bandits for Learning to Explain from Explanations](https://mlsanity.com/paper/2102.03815/) (2021)
- [Efficient Encrypted Inference on Ensembles of Decision Trees](https://mlsanity.com/paper/2103.03411/) (2021)
- [SONG: Self-Organizing Neural Graphs](https://mlsanity.com/paper/2107.13214/) (2021)
- [Explaining Any ML Model? -- On Goals and Capabilities of XAI](https://mlsanity.com/paper/2206.13888/) (2022)
- [The Shape of Explanations: A Topological Account of Rule-Based Explanations in Machine Learning](https://mlsanity.com/paper/2301.09042/) (2023)
- [Interpreting GNN-based IDS Detections Using Provenance Graph Structural Features](https://mlsanity.com/paper/2306.00934/) (2023)
- [Conditional Adversarial Fragility in Financial Machine Learning under Macroeconomic Stress](https://mlsanity.com/paper/2512.19935/) (2025)
- [No Certificate for Alignment: Two Independent Impossibilities and the Pareto Frontier of Achievable Safety Guarantees](https://mlsanity.com/paper/2603.08761/) (2026)
- [Assessing Model-Agnostic XAI Methods against EU AI Act Explainability Requirements](https://mlsanity.com/paper/2604.09628/) (2026)
- [Trustworthy Clinical Decision Support Using Meta-Predicates and Domain-Specific Languages](https://mlsanity.com/paper/2604.21263/) (2026)
- [Generalized Adversarial Distances to Efficiently Discover Classifier Errors](https://mlsanity.com/paper/2102.12844/) (2021)
- [Information-theoretic Evolution of Model Agnostic Global Explanations](https://mlsanity.com/paper/2105.06956/) (2021)
- [Multihop: Leveraging Complex Models to Learn Accurate Simple Models](https://mlsanity.com/paper/2109.06961/) (2021)

## Abstract (arXiv)

Interpretability has become incredibly important as machine learning is increasingly used to inform consequential decisions. We propose to construct global explanations of complex, blackbox models in the form of a decision tree approximating the original model---as long as the decision tree is a good approximation, then it mirrors the computation performed by the blackbox model. We devise a novel algorithm for extracting decision tree explanations that actively samples new training points to avoid overfitting. We evaluate our algorithm on a random forest to predict diabetes risk and a learned controller for cart-pole. Compared to several baselines, our decision trees are both substantially more accurate and equally or more interpretable based on a user study. Finally, we describe several insights provided by our interpretations, including a causal issue validated by a physician.