mlsanityData to 7 Oct 2026

Model Reconstruction from Model Explanations

arXiv 1807.05185 ↗First posted 13 Jul 2018stat.ML cs.LGcited by 84

Not on the map. Why?

Abstract (arXiv)

We show through theory and experiment that gradient-based explanations of a model quickly reveal the model itself. Our results speak to a tension between the desire to keep a proprietary model secret and the ability to offer model explanations. On the theoretical side, we give an algorithm that provably learns a two-layer ReLU network in a setting where the algorithm may query the gradient of the model with respect to chosen inputs. The number of queries is independent of the dimension and nearly optimal in its dependence on the model size. Of interest not only from a learning-theoretic perspective, this result highlights the power of gradients rather than labels as a learning primitive. Complementing our theory, we give effective heuristics for reconstructing models from gradient explanations that are orders of magnitude more query-efficient than reconstruction attacks relying on prediction interfaces.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 16; 2023: 18; 2024: 14; 2025: 13; 2026: 8.

84 papers.

Sort:

By area:

  1. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  2. 2022Reconstructing Training Data from Trained Neural Networkscited by 129
  3. 2022I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defencescited by 75
  4. 2022Towards Data-Free Model Stealing in a Hard Label Settingcited by 57
  5. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 52
  6. 2021Exploiting Explanations for Model Inversion Attackscited by 48
  7. 2021Survey: Leakage and Privacy at Inference Timecited by 44
  8. 2021DeepSteal: Advanced Model Extractions Leveraging Efficient Weight Stealing in Memoriescited by 42
  9. 2023Hidden symmetries of ReLU networkscited by 38
  10. 2021MEGEX: Data-Free Model Extraction Attack against Gradient-Based Explainable AIcited by 35
  11. 2024Gradient based Feature Attribution in Explainable AI: A Technical Reviewcited by 34
  12. 2022DualCF: Efficient Model Extraction Attack from Counterfactual Explanationscited by 31
38 more
  1. 2022Hardness of Noise-Free Learning for Two-Hidden-Layer Neural Networkscited by 27
  2. 2021Characterizing the risk of fairwashingcited by 21
  3. 2021Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloudcited by 19
  4. 2023Stealing the Decoding Algorithms of Language Modelscited by 19
  5. 2023Polynomial Time Cryptanalytic Extraction of Neural Network Modelscited by 17
  6. 2022Inferring Sensitive Attributes from Model Explanationscited by 15
  7. 2022Differentially Private Counterfactuals via Functional Mechanismcited by 13
  8. 2024Model Reconstruction Using Counterfactual Explanations: A Perspective From Polytope Theorycited by 12
  9. 2021An Exact Poly-Time Membership-Queries Algorithm for Extraction a three-Layer ReLU Networkcited by 11
  10. 2022Feature-based Learning for Diverse and Privacy-Preserving Counterfactual Explanationscited by 11
  11. 2022XRand: Differentially Private Defense against Explanation-Guided Attackscited by 11
  12. 2023Marich: A Query-efficient Distributionally Equivalent Model Extraction Attack using Public Datacited by 11
  13. 2023Isolation and Induction: Training Robust Deep Neural Networks against Model Stealing Attackscited by 10
  14. 2021SoK: Machine Learning Governancecited by 9
  15. 2023AUTOLYCUS: Exploiting Explainable AI (XAI) for Model Extraction Attacks against Interpretable Modelscited by 9
  16. 2023A Survey on Explainable Artificial Intelligence for Cybersecuritycited by 9
  17. 2024Feature Inference Attack on Shapley Valuescited by 9
  18. 2021Model Extraction and Defenses on Generative Adversarial Networkscited by 8
  19. 2022On the amplification of security and privacy risks by post-hoc explanations in machine learning modelscited by 8
  20. 2024Polynomial Time Cryptanalytic Extraction of Deep Neural Networks in the Hard-Label Settingcited by 8
  21. 2022SoK: Modeling Explainability in Security Analytics for Interpretability, Trustworthiness, and Usabilitycited by 7
  22. 2024Beyond Slow Signs in High-fidelity Model Extractioncited by 7
  23. 2021Key Technology Considerations in Developing and Deploying Machine Learning Models in Clinical Radiology Practicecited by 6
  24. 2021Efficiently Learning Any One Hidden Layer ReLU Network From Queriescited by 6
  25. 2021Evaluating Privacy-Preserving Machine Learning in Critical Infrastructures: A Case Study on Time-Series Classificationcited by 6
  26. 2023MeaeQ: Mount Model Extraction Attacks with Efficient Queriescited by 6
  27. 2024A Survey of Privacy-Preserving Model Explanations: Privacy Risks, Attacks, and Countermeasurescited by 6
  28. 2022Preprocessors Matter! Realistic Decision-Based Attacks on Machine Learning Systemscited by 5
  29. 2023Model Extraction Attacks on Split Federated Learningcited by 5
  30. 2023Accurate, Explainable, and Private Models: Providing Recourse While Minimizing Training Data Leakagecited by 5
  31. 2023Privacy-Preserving Algorithmic Recoursecited by 5
  32. 2022Privacy of Autonomous Vehicles: Risks, Protection Methods, and Future Directionscited by 4
  33. 2023Toward L_∞-recovery of Nonlinear Functions: A Polynomial Sample Complexity Bound for Gaussian Random Fieldscited by 4
  34. 2023SoK: Taming the Triangle -- On the Interplays between Fairness, Interpretability and Privacy in Machine Learningcited by 4
  35. 2025Crowding Out The Noise: Algorithmic Collective Action Under Differential Privacycited by 4
  36. 2021EG-Booster: Explanation-Guided Booster of ML Evasion Attackscited by 3
  37. 2022Margin-distancing for safe model explanationcited by 3
  38. 2024Privacy Implications of Explainable AI in Data-Driven Systemscited by 3

50 most cited of 84 All 84 (JSON)