mlsanityData to 7 Oct 2026

Foundations of Interpretable Models

arXiv 2508.00545 ↗First posted 1 Aug 2025cs.LG cs.AI cs.NEcited by 3

Not on the map. Why?

Abstract (arXiv)

We argue that existing definitions of interpretability are not actionable in that they fail to inform users about general, sound, and robust interpretable model design. This makes current interpretability research fundamentally ill-posed. To address this issue, we propose a definition of interpretability that is general, simple, and subsumes existing informal notions within the interpretable AI community. We show that our definition is actionable, as it directly reveals the foundational properties, underlying assumptions, principles, data structures, and architectural features necessary for designing interpretable models. Building on this, we propose a general blueprint for designing interpretable models and introduce the first open-sourced library with native support for interpretable data structures and processes.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 34 of its 72 references that are in the map. First posted on arXiv 1 Aug 2025.

Builds on

34 of 72 references.

References · 34

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  6. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  7. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  8. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  9. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  10. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  11. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  12. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
22 more
  1. 2020Concept Bottleneck Modelscited by 1,022
  2. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  3. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  4. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  5. 2017Exploring Generalization in Deep Learningcited by 595
  6. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  7. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  8. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
  9. 2017Generalization in Deep Learningcited by 146
  10. 2019Explaining Classifiers with Causal Concept Effect (CaCE)cited by 135
  11. 2021Promises and Pitfalls of Black-Box Concept Learning Modelscited by 128
  12. 2023The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learningcited by 63
  13. 2023Learning to Receive Help: Intervention-Aware Concept Embedding Modelscited by 41
  14. 2023Human Uncertainty in Concept-Based AI Systemscited by 39
  15. 2023Towards Robust Metrics for Concept Representation Evaluationcited by 29
  16. 2024Beyond Concept Bottleneck Models: How to Make Black Boxes Intervenable?cited by 28
  17. 2024Causal Concept Graph Models: Beyond Causal Opacity in Deep Learningcited by 28
  18. 2025We Can't Understand AI Using our Existing Vocabularycited by 25
  19. 2023Interpretability is in the Mind of the Beholder: A Causal Framework for Human-interpretable Representation Learningcited by 22
  20. 2024A Complexity-Based Theory of Compositionalitycited by 13
  21. 2023Selective Concept Models: Permitting Stakeholder Customisation at Test-Timecited by 6
  22. 2025Interpretable Hierarchical Concept Reasoning through Attention-Guided Graph Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026From Mechanistic to Compositional Interpretabilitycited by 1
  2. 2026Interpretability in Deep Time Series Models Demands Semantic Alignmentcited by 0
  3. 2026Graphical Design of Interpretable Architecturescited by 0