mlsanityData to 14 Oct 2026

Categorical Foundations of Explainable AI: A Unifying Theory

arXiv 2304.14094 ↗First posted 27 Apr 2023cs.AI cs.LG stat.MLcited by 7

Not on the map. Why?

Abstract (arXiv)

Explainable AI (XAI) aims to address the human need for safe and reliable AI systems. However, numerous surveys emphasize the absence of a sound mathematical formalization of key XAI notions -- remarkably including the term "explanation" which still lacks a precise definition. To bridge this gap, this paper presents the first mathematically rigorous definitions of key XAI notions and processes, using the well-funded formalism of Category theory. We show that our categorical framework allows to: (i) model existing learning schemes and architectures, (ii) formally define the term "explanation", (iii) establish a theoretical basis for XAI taxonomies, and (iv) analyze commonly overlooked aspects of explaining methods. As a consequence, our categorical framework promotes the ethical and secure deployment of AI technologies as it represents a significant step towards a sound theoretical foundation of explainable AI.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 38 of its 89 references that are in the map. First posted on arXiv 27 Apr 2023.

Builds on

38 of 89 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  6. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  7. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  8. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  9. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  10. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  11. 2020Concept Bottleneck Modelscited by 1,032
  12. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 838
26 more
  1. 2017Interpretation of Neural Networks is Fragilecited by 444
  2. 2019Neural Probabilistic Logic Programming in DeepProbLogcited by 394
  3. 2020Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Surveycited by 254
  4. 2016Model-Agnostic Interpretability of Machine Learningcited by 219
  5. 2020A Survey of the State of Explainable AI for Natural Language Processingcited by 206
  6. 2020The Next Decade in AI: Four Steps Towards Robust Artificial Intelligencecited by 195
  7. 2017Visualizing Deep Neural Network Decisions: Prediction Difference Analysiscited by 166
  8. 2018Local Rule-Based Explanations of Black Box Decision Systemscited by 166
  9. 2015Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction modelcited by 140
  10. 2020Drug discovery with explainable artificial intelligencecited by 118
  11. 2016Scalable Bayesian Rule Listscited by 64
  12. 2019A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statisticscited by 56
  13. 2019Learning Symbolic Physics with Graph Networkscited by 53
  14. 2021Logic Explained Networkscited by 45
  15. 2021Categorical Foundations of Gradient-Based Learningcited by 44
  16. 2021Category Theory in Machine Learningcited by 36
  17. 2022A Survey of Explainable Graph Neural Networks: Taxonomy and Evaluation Metricscited by 26
  18. 2022Explainable AI for clinical and remote health applications: a survey on tabular and time series datacited by 24
  19. 2017Automatic Generation of Natural Language Explanationscited by 22
  20. 2021XAI Handbook: Towards a Unified Framework for Explainable AIcited by 14
  21. 2022Learnable Commutative Monoids for Graph Neural Networkscited by 13
  22. 2021Reverse Derivative Ascent: A Categorical Approach to Learning Boolean Circuitscited by 10
  23. 2014Graph-Sparse LDA: A Topic Model with Structured Sparsitycited by 7
  24. 2019Differentiable Causal Computations via Delayed Tracecited by 5
  25. 2021Compositional Semantics for Probabilistic Programs with Exact Conditioningcited by 2
  26. 2022Monoidal Streams for Dataflow Programmingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 4.

7 papers.

  1. 2024Towards Compositional Interpretability for XAIcited by 12
  2. 2025Mathematical Foundation of Interpretable Equivariant Surrogate Modelscited by 5
  3. 2026Actionable Interpretability Must Be Defined in Terms of Symmetriescited by 4
  4. 2025Logic Explanation of AI Classifiers by Categorical Explaining Functorscited by 1
  5. 2026Interpretability in Deep Time Series Models Demands Semantic Alignmentcited by 0
  6. 2026Graphical Design of Interpretable Architecturescited by 0
  7. 2026A Comparative Explainability Framework for DeBERTa-v3 in Zero-Shot Medical Abstract Classificationcited by 0