# Energy-Based Concept Bottleneck Models: Unifying Prediction, Concept Intervention, and Probabilistic Interpretations

arXiv: https://arxiv.org/abs/2401.14142 · HTML: https://mlsanity.com/paper/2401.14142/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 50 references that are in the map.
- First posted on arXiv 25 Jan 2024.

## Builds on (most cited first, 30 of 40)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One](https://mlsanity.com/paper/1912.03263/) (2019)
- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification](https://mlsanity.com/paper/2211.11158/) (2022)
- [Post-hoc Concept Bottleneck Models](https://mlsanity.com/paper/2205.15480/) (2022)
- [Label-Free Concept Bottleneck Models](https://mlsanity.com/paper/2304.06129/) (2023)
- [A Theory of Generative ConvNet](https://mlsanity.com/paper/1602.03264/) (2016)
- [On the Anatomy of MCMC-Based Maximum Likelihood Learning of Energy-Based Models](https://mlsanity.com/paper/1903.12370/) (2019)
- [Probabilistic Concept Bottleneck Models](https://mlsanity.com/paper/2306.01574/) (2023)
- [Residual Energy-Based Models for Text Generation](https://mlsanity.com/paper/2004.11714/) (2020)
- [Continuously Indexed Domain Adaptation](https://mlsanity.com/paper/2007.01807/) (2020)
- [Unsupervised Learning of Compositional Energy Concepts](https://mlsanity.com/paper/2111.03042/) (2021)
- [Concept Embedding Models: Beyond the Accuracy-Explainability Trade-Off](https://mlsanity.com/paper/2209.09056/) (2022)
- [Cooperative Training of Descriptor and Generator Networks](https://mlsanity.com/paper/1609.09408/) (2016)
- [Structured Prediction Energy Networks](https://mlsanity.com/paper/1511.06350/) (2015)
- [Towards Bayesian Deep Learning: A Framework and Some Existing Methods](https://mlsanity.com/paper/1608.06884/) (2016)
- [Interpretable Neural-Symbolic Concept Reasoning](https://mlsanity.com/paper/2304.14068/) (2023)
- [Now You See Me (CME): Concept-based Model Extraction](https://mlsanity.com/paper/2010.13233/) (2020)
- [Entropy-based Logic Explanations of Neural Networks](https://mlsanity.com/paper/2106.06804/) (2021)
- [A Framework for Learning Ante-hoc Explainable Models via Concepts](https://mlsanity.com/paper/2108.11761/) (2021)
- [Unsupervised Compositional Concepts Discovery with Text-to-Image Generative Models](https://mlsanity.com/paper/2306.05357/) (2023)
- [This Looks Like Those: Illuminating Prototypical Concepts Using Multiple Visualizations](https://mlsanity.com/paper/2310.18589/) (2023)
- [ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2005.00850/) (2020)
- [OrphicX: A Causality-Inspired Latent Variable Model for Interpreting Graph Neural Networks](https://mlsanity.com/paper/2203.15209/) (2022)
- [Energy-Based Models for Continual Learning](https://mlsanity.com/paper/2011.12216/) (2020)

## Built on by (most cited first, 30 of 33)

- [Concept Bottleneck Large Language Models](https://mlsanity.com/paper/2412.07992/) (2024)
- [DCBM: Data-Efficient Visual Concept Bottleneck Models](https://mlsanity.com/paper/2412.11576/) (2024)
- [Leakage and Interpretability in Concept-Based Models](https://mlsanity.com/paper/2504.14094/) (2025)
- [VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a Unified Concept Set](https://mlsanity.com/paper/2510.21323/) (2025)
- [V2C-CBM: Building Concept Bottlenecks with Vision-to-Concept Tokenizer](https://mlsanity.com/paper/2501.04975/) (2025)
- [Graph Concept Bottleneck Models](https://mlsanity.com/paper/2508.14255/) (2025)
- [There Was Never a Bottleneck in Concept Bottleneck Models](https://mlsanity.com/paper/2506.04877/) (2025)
- [Probabilistic Conceptual Explainers: Trustworthy Conceptual Explanations for Vision Foundation Models](https://mlsanity.com/paper/2406.12649/) (2024)
- [An Analysis of Concept Bottleneck Models: Measuring, Understanding, and Mitigating the Impact of Noisy Annotations](https://mlsanity.com/paper/2505.16705/) (2025)
- [LogicCBMs: Logic-Enhanced Concept-Based Learning](https://mlsanity.com/paper/2512.07383/) (2025)
- [Locality-aware Concept Bottleneck Model](https://mlsanity.com/paper/2508.14562/) (2025)
- [Concept Complement Bottleneck Model for Interpretable Medical Image Diagnosis](https://mlsanity.com/paper/2410.15446/) (2024)
- [Birds look like cars: Adversarial analysis of intrinsically interpretable deep learning](https://mlsanity.com/paper/2503.08636/) (2025)
- [Towards Reasonable Concept Bottleneck Models](https://mlsanity.com/paper/2506.05014/) (2025)
- [Hierarchical Concept-based Interpretable Models](https://mlsanity.com/paper/2602.23947/) (2026)
- [Factor Graph-based Interpretable Neural Networks](https://mlsanity.com/paper/2502.14572/) (2025)
- [Adaptive Test-Time Intervention for Concept Bottleneck Models](https://mlsanity.com/paper/2503.06730/) (2025)
- [AIM: Amending Inherent Interpretability via Self-Supervised Masking](https://mlsanity.com/paper/2508.11502/) (2025)
- [Concept-Guided Backdoor Attack on Vision Language Models](https://mlsanity.com/paper/2512.00713/) (2025)
- [KLDrive: Fine-Grained 3D Scene Reasoning for Autonomous Driving based on Knowledge Graph](https://mlsanity.com/paper/2603.21029/) (2026)
- [Matryoshka Concept Bottleneck Models](https://mlsanity.com/paper/2605.20612/) (2026)
- [Parameter-Efficient Construction of the Rashomon Slice for Concept Bottleneck Models](https://mlsanity.com/paper/2511.19636/) (2025)
- [Do Neural Networks Preserve Case Structure? Case-Based Decomposition, Interpretation, and Decision Consistency](https://mlsanity.com/paper/2607.11347/) (2026)
- [Bi-ICE: An Inner Interpretable Framework for Image Classification via Bi-directional Interactions between Concept and Input Embeddings](https://mlsanity.com/paper/2411.18645/) (2024)
- [CoBELa: Steering Transparent Generation via Concept Bottlenecks on Energy Landscapes](https://mlsanity.com/paper/2507.08334/) (2025)
- [Multimodal Attention-Aware Fusion for Diagnosing Distal Myopathy: Evaluating Model Interpretability and Clinician Trust](https://mlsanity.com/paper/2508.01316/) (2025)
- [Chat-CBM: Towards Interactive Concept Bottleneck Models with Frozen Large Language Models](https://mlsanity.com/paper/2509.17522/) (2025)
- [Information Theory in Open-world Machine Learning Foundations, Frameworks, and Future Direction](https://mlsanity.com/paper/2510.15422/) (2025)
- [Exploring Concept Subspace for Self-explainable Text-Attributed Graph Learning](https://mlsanity.com/paper/2604.11986/) (2026)
- [SynCB: A Synergy Concept-Based Model with Dynamic Routing Between Concepts and Complementary Neural Branches](https://mlsanity.com/paper/2605.20908/) (2026)

## Abstract (arXiv)

Existing methods, such as concept bottleneck models (CBMs), have been successful in providing concept-based interpretations for black-box deep learning models. They typically work by predicting concepts given the input and then predicting the final class label given the predicted concepts. However, (1) they often fail to capture the high-order, nonlinear interaction between concepts, e.g., correcting a predicted concept (e.g., "yellow breast") does not help correct highly correlated concepts (e.g., "yellow belly"), leading to suboptimal final accuracy; (2) they cannot naturally quantify the complex conditional dependencies between different concepts and class labels (e.g., for an image with the class label "Kentucky Warbler" and a concept "black bill", what is the probability that the model correctly predicts another concept "black crown"), therefore failing to provide deeper insight into how a black-box model works. In response to these limitations, we propose Energy-based Concept Bottleneck Models (ECBMs). Our ECBMs use a set of neural networks to define the joint energy of candidate (input, concept, class) tuples. With such a unified interface, prediction, concept correction, and conditional dependency quantification are then represented as conditional probabilities, which are generated by composing different energy functions. Our ECBMs address both limitations of existing CBMs, providing higher accuracy and richer concept interpretations. Empirical results show that our approach outperforms the state-of-the-art on real-world datasets.