mlsanityData to 7 Oct 2026

Actionable Interpretability Must Be Defined in Terms of Symmetries

arXiv 2601.12913 ↗First posted 19 Jan 2026cs.AI cs.LG cs.NEcited by 4

Not on the map. Why?

Abstract (arXiv)

This paper argues that interpretability research in Artificial Intelligence (AI) is fundamentally ill-posed as existing definitions of interpretability fail to describe how interpretability can be formally tested or designed for. We posit that actionable definitions of interpretability must be formulated in terms of *symmetries* that inform model design and lead to testable conditions. Under a probabilistic view, we hypothesise that four symmetries (inference equivariance, information invariance, concept-closure invariance, and structural invariance) suffice to (i) formalise interpretable models as a subclass of probabilistic models, (ii) yield a unified formulation of interpretable inference (e.g., alignment, interventions, and counterfactuals) as a form of Bayesian inversion, and (iii) provide a formal framework to verify compliance with safety standards and regulations.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 35 of its 70 references that are in the map. First posted on arXiv 19 Jan 2026.

Builds on

35 of 70 references.

References · 35

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  4. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  5. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  6. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  7. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  8. 2020Concept Bottleneck Modelscited by 1,022
  9. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  10. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  11. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  12. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
23 more
  1. 2020Neural Additive Models: Interpretable Machine Learning with Neural Netscited by 318
  2. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  3. 2019FACE: Feasible and Actionable Counterfactual Explanationscited by 232
  4. 2020The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Modelscited by 125
  5. 2019Optimal Sparse Decision Treescited by 104
  6. 2023Concept-based Explainable Artificial Intelligence: A Surveycited by 97
  7. 2022Interactive Concept Bottleneck Modelscited by 79
  8. 2023A Closer Look at the Intervention Procedure of Concept Bottleneck Modelscited by 63
  9. 2019A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statisticscited by 55
  10. 2023Learning to Receive Help: Intervention-Aware Concept Embedding Modelscited by 41
  11. 2023Human Uncertainty in Concept-Based AI Systemscited by 39
  12. 2017Disintegration and Bayesian Inversion via String Diagramscited by 32
  13. 2024Beyond Concept Bottleneck Models: How to Make Black Boxes Intervenable?cited by 28
  14. 2024Causal Concept Graph Models: Beyond Causal Opacity in Deep Learningcited by 28
  15. 2023Interpretability is in the Mind of the Beholder: A Causal Framework for Human-interpretable Representation Learningcited by 22
  16. 2024Interpretable Concept-Based Memory Reasoningcited by 17
  17. 2024Towards Compositional Interpretability for XAIcited by 10
  18. 2023Categorical Foundations of Explainable AI: A Unifying Theorycited by 7
  19. 2023Selective Concept Models: Permitting Stakeholder Customisation at Test-Timecited by 6
  20. 2025Mathematical Foundation of Interpretable Equivariant Surrogate Modelscited by 5
  21. 2025Partial Markov Categoriescited by 2
  22. 2025Logic Explanation of AI Classifiers by Categorical Explaining Functorscited by 1
  23. 2025Compositional Inference for Bayesian Networks and Causalitycited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Causality is Key for Interpretability Claims to Generalisecited by 8
  2. 2026From Mechanistic to Compositional Interpretabilitycited by 1
  3. 2026TRACE: A Concept Bottleneck Model for Longitudinal 3D Glioblastoma Response Assessmentcited by 0
  4. 2026LAWFUL: Law-Aligned Witness for Faithful Use of Latentscited by 0