# Causal Concept Graph Models: Beyond Causal Opacity in Deep Learning

arXiv: https://arxiv.org/abs/2405.16507 · HTML: https://mlsanity.com/paper/2405.16507/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 45 references that are in the map.
- First posted on arXiv 26 May 2024.

## Builds on (most cited first, 29 of 29)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [Label-Free Concept Bottleneck Models](https://mlsanity.com/paper/2304.06129/) (2023)
- [On Completeness-aware Concept-Based Explanations in Deep Neural Networks](https://mlsanity.com/paper/1910.07969/) (2019)
- [Concept Whitening for Interpretable Image Recognition](https://mlsanity.com/paper/2002.01650/) (2020)
- [Explaining Classifiers with Causal Concept Effect (CaCE)](https://mlsanity.com/paper/1907.07165/) (2019)
- [Promises and Pitfalls of Black-Box Concept Learning Models](https://mlsanity.com/paper/2106.13314/) (2021)
- [Fair Attribute Classification through Latent Space De-biasing](https://mlsanity.com/paper/2012.01469/) (2020)
- [Causal Machine Learning: A Survey and Open Problems](https://mlsanity.com/paper/2206.15475/) (2022)
- [Probabilistic Concept Bottleneck Models](https://mlsanity.com/paper/2306.01574/) (2023)
- [Probabilities of Causation: Bounds and Identification](https://mlsanity.com/paper/1301.3898/) (2013)
- [Neural Network Attributions: A Causal Perspective](https://mlsanity.com/paper/1902.02302/) (2019)
- [Greedy Relaxations of the Sparsest Permutation Algorithm](https://mlsanity.com/paper/2206.05421/) (2022)
- [A Closer Look at the Intervention Procedure of Concept Bottleneck Models](https://mlsanity.com/paper/2302.14260/) (2023)
- [Concept Embedding Models: Beyond the Accuracy-Explainability Trade-Off](https://mlsanity.com/paper/2209.09056/) (2022)
- [Interpretable Neural-Symbolic Concept Reasoning](https://mlsanity.com/paper/2304.14068/) (2023)
- [Entropy-based Logic Explanations of Neural Networks](https://mlsanity.com/paper/2106.06804/) (2021)
- [CausalVAE: Structured Causal Disentanglement in Variational Autoencoder](https://mlsanity.com/paper/2004.08697/) (2020)
- [Logic Explained Networks](https://mlsanity.com/paper/2108.05149/) (2021)
- [Causal Explanations and XAI](https://mlsanity.com/paper/2201.13169/) (2022)
- [Relational Concept Bottleneck Models](https://mlsanity.com/paper/2308.11991/) (2023)
- [Interpretable and intervenable ultrasonography-based machine learning models for pediatric appendicitis](https://mlsanity.com/paper/2302.14460/) (2023)
- [DiConStruct: Causal Concept-based Explanations through Black-Box Distillation](https://mlsanity.com/paper/2401.08534/) (2024)
- [On the Foundations of Cycles in Bayesian Networks](https://mlsanity.com/paper/2301.08608/) (2023)

## Built on by (most cited first, 29 of 29)

- [A Comprehensive Survey on Self-Interpretable Neural Networks](https://mlsanity.com/paper/2501.15638/) (2025)
- [Causally Reliable Concept Bottleneck Models](https://mlsanity.com/paper/2503.04363/) (2025)
- [If Concept Bottlenecks are the Question, are Foundation Models the Answer?](https://mlsanity.com/paper/2504.19774/) (2025)
- [Neural Probabilistic Circuits: Enabling Compositional and Interpretable Predictions through Logical Reasoning](https://mlsanity.com/paper/2501.07021/) (2025)
- [Foundations of Interpretable Models](https://mlsanity.com/paper/2508.00545/) (2025)
- [Actionable Interpretability Must Be Defined in Terms of Symmetries](https://mlsanity.com/paper/2601.12913/) (2026)
- [V-CEM: Bridging Performance and Intervenability in Concept-based Models](https://mlsanity.com/paper/2504.03978/) (2025)
- [Towards Reasonable Concept Bottleneck Models](https://mlsanity.com/paper/2506.05014/) (2025)
- [Interpretable Hierarchical Concept Reasoning through Attention-Guided Graph Learning](https://mlsanity.com/paper/2506.21102/) (2025)
- [Quantifying the Accuracy-Interpretability Trade-Off in Concept-Based Sidechannel Models](https://mlsanity.com/paper/2510.05670/) (2025)
- [Hierarchical Concept-based Interpretable Models](https://mlsanity.com/paper/2602.23947/) (2026)
- [Mixture of Concept Bottleneck Experts](https://mlsanity.com/paper/2602.02886/) (2026)
- [Prototype-Grounded Concept Models for Verifiable Concept Alignment](https://mlsanity.com/paper/2604.16076/) (2026)
- [Logic Explanation of AI Classifiers by Categorical Explaining Functors](https://mlsanity.com/paper/2503.16203/) (2025)
- [Flexible Concept Bottleneck Model](https://mlsanity.com/paper/2511.06678/) (2025)
- [The Standard Interpretable Model: A general theory of interpretable machine learning to deductively design interpretable methods using Lagrangian mechanics](https://mlsanity.com/paper/2606.12289/) (2026)
- [Intervening in Black Box: Concept Bottleneck Model for Enhancing Human Neural Network Mutual Understanding](https://mlsanity.com/paper/2506.22803/) (2025)
- [Neural Concept Verifier: Scaling Prover-Verifier Games via Concept Encodings](https://mlsanity.com/paper/2507.07532/) (2025)
- [From Black-box to Causal-box: Towards Building More Interpretable Models](https://mlsanity.com/paper/2510.21998/) (2025)
- [Interpretability in Deep Time Series Models Demands Semantic Alignment](https://mlsanity.com/paper/2602.02239/) (2026)
- [Federated Concept-Based Models: Interpretable models with distributed supervision](https://mlsanity.com/paper/2602.04093/) (2026)
- [Causal Neural Probabilistic Circuits](https://mlsanity.com/paper/2603.01372/) (2026)
- [Digging Deeper: Learning Multi-Level Concept Hierarchies](https://mlsanity.com/paper/2603.10084/) (2026)
- [Concept Graph Convolutions: Message Passing in the Concept Space](https://mlsanity.com/paper/2604.20082/) (2026)
- [Measuring What Matters: Synthetic Benchmarks for Concept Bottleneck Models](https://mlsanity.com/paper/2606.04326/) (2026)
- [Feature Attribution in Directed Acyclic Graphs Using Edge Intervention](https://mlsanity.com/paper/2606.15273/) (2026)
- [Assessing Reliability of Symbol Detection in Concept Bottleneck Models](https://mlsanity.com/paper/2606.16535/) (2026)
- [Concept Flow Models: Anchoring Concept-Based Reasoning with Hierarchical Bottlenecks](https://mlsanity.com/paper/2606.19489/) (2026)
- [Concept-based Visual Counterfactual Explanations with Diffusion Models](https://mlsanity.com/paper/2607.22544/) (2026)

## Abstract (arXiv)

Causal opacity denotes the difficulty in understanding the "hidden" causal structure underlying the decisions of deep neural network (DNN) models. This leads to the inability to rely on and verify state-of-the-art DNN-based systems, especially in high-stakes scenarios. For this reason, circumventing causal opacity in DNNs represents a key open challenge at the intersection of deep learning, interpretability, and causality. This work addresses this gap by introducing Causal Concept Graph Models (Causal CGMs), a class of interpretable models whose decision-making process is causally transparent by design. Our experiments show that Causal CGMs can: (i) match the generalisation performance of causally opaque models, (ii) enable human-in-the-loop corrections to mispredicted intermediate reasoning steps, boosting not just downstream accuracy after corrections but also the reliability of the explanations provided for specific instances, and (iii) support the analysis of interventional and counterfactual scenarios, thereby improving the model's causal interpretability and supporting the effective verification of its reliability and fairness.