# Understanding and Enhancing Robustness of Concept-based Models

arXiv: https://arxiv.org/abs/2211.16080 · HTML: https://mlsanity.com/paper/2211.16080/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 38 references that are in the map.
- First posted on arXiv 29 Nov 2022.

## Builds on (most cited first, 23 of 23)

- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [The (Un)reliability of saliency methods](https://mlsanity.com/paper/1711.00867/) (2017)
- [On Completeness-aware Concept-Based Explanations in Deep Neural Networks](https://mlsanity.com/paper/1910.07969/) (2019)
- [Explanations can be manipulated and geometry is to blame](https://mlsanity.com/paper/1906.07983/) (2019)
- [Learning Deep Representations of Fine-grained Visual Descriptions](https://mlsanity.com/paper/1605.05395/) (2016)
- [Explaining Classifiers with Causal Concept Effect (CaCE)](https://mlsanity.com/paper/1907.07165/) (2019)
- [Promises and Pitfalls of Black-Box Concept Learning Models](https://mlsanity.com/paper/2106.13314/) (2021)
- [Do Concept Bottleneck Models Learn as Intended?](https://mlsanity.com/paper/2105.04289/) (2021)
- [Concept Bottleneck Model with Additional Unsupervised Concepts](https://mlsanity.com/paper/2202.01459/) (2022)
- [Robust and Stable Black Box Explanations](https://mlsanity.com/paper/2011.06169/) (2020)
- [Certifiably Robust Interpretation in Deep Learning](https://mlsanity.com/paper/1905.12105/) (2019)
- [Perturbing Inputs for Fragile Interpretations in Deep Natural Language Processing](https://mlsanity.com/paper/2108.04990/) (2021)
- [ExpBERT: Representation Engineering with Natural Language Explanations](https://mlsanity.com/paper/2005.01932/) (2020)
- [Explaining Neural Networks Semantically and Quantitatively](https://mlsanity.com/paper/1812.07169/) (2018)
- [Adversarial TCAV -- Robust and Effective Interpretation of Intermediate Layers in Neural Networks](https://mlsanity.com/paper/2002.03549/) (2020)
- [On Saliency Maps and Adversarial Robustness](https://mlsanity.com/paper/2006.07828/) (2020)
- [Automatic Concept Extraction for Concept Bottleneck-based Video Classification](https://mlsanity.com/paper/2206.10129/) (2022)

## Built on by (most cited first, 22 of 22)

- [Concept-based Explainable Artificial Intelligence: A Survey](https://mlsanity.com/paper/2312.12936/) (2023)
- [Leakage and Interpretability in Concept-Based Models](https://mlsanity.com/paper/2504.14094/) (2025)
- [Do Concept Bottleneck Models Respect Localities?](https://mlsanity.com/paper/2401.01259/) (2024)
- [Towards Modeling Uncertainties of Self-explaining Neural Networks via Conformal Prediction](https://mlsanity.com/paper/2401.01549/) (2024)
- [Evaluating Readability and Faithfulness of Concept-based Explanations](https://mlsanity.com/paper/2404.18533/) (2024)
- [A survey on Concept-based Approaches For Model Improvement](https://mlsanity.com/paper/2403.14566/) (2024)
- [Towards Multi-dimensional Explanation Alignment for Medical Classification](https://mlsanity.com/paper/2410.21494/) (2024)
- [SUB: Benchmarking CBM Generalization via Synthetic Attribute Substitutions](https://mlsanity.com/paper/2507.23784/) (2025)
- [An Analysis of Concept Bottleneck Models: Measuring, Understanding, and Mitigating the Impact of Noisy Annotations](https://mlsanity.com/paper/2505.16705/) (2025)
- [LogicCBMs: Logic-Enhanced Concept-Based Learning](https://mlsanity.com/paper/2512.07383/) (2025)
- [A Comprehensive Survey on the Risks and Limitations of Concept-based Models](https://mlsanity.com/paper/2506.04237/) (2025)
- [Enhance Diffusion to Improve Robust Generalization](https://mlsanity.com/paper/2306.02618/) (2023)
- [A Self-explaining Neural Architecture for Generalizable Concept Learning](https://mlsanity.com/paper/2405.00349/) (2024)
- [Birds look like cars: Adversarial analysis of intrinsically interpretable deep learning](https://mlsanity.com/paper/2503.08636/) (2025)
- [CoLiDR: Concept Learning using Aggregated Disentangled Representations](https://mlsanity.com/paper/2407.19300/) (2024)
- [Factor Graph-based Interpretable Neural Networks](https://mlsanity.com/paper/2502.14572/) (2025)
- [ASCENT-ViT: Attention-based Scale-aware Concept Learning Framework for Enhanced Alignment in Vision Transformers](https://mlsanity.com/paper/2501.09221/) (2025)
- [Understanding and Improving Adversarial Robustness of Neural Probabilistic Circuits](https://mlsanity.com/paper/2509.20549/) (2025)
- [Structural Causality-based Generalizable Concept Discovery Models](https://mlsanity.com/paper/2410.15491/) (2024)
- [Concept Labels Are Not Enough: Rethinking Concept Bottleneck Models through Representation Integrity](https://mlsanity.com/paper/2510.15770/) (2025)
- [Formal Concept Lattices are Good Semantic Scaffolds for Concept-Based Learning](https://mlsanity.com/paper/2606.05471/) (2026)
- [Interpretable but Fragile? Robustness of Concept Bottlenecks under Geometric-Semantic Perturbations](https://mlsanity.com/paper/2609.38625/) (2026)

## Abstract (arXiv)

Rising usage of deep neural networks to perform decision making in critical applications like medical diagnosis and financial analysis have raised concerns regarding their reliability and trustworthiness. As automated systems become more mainstream, it is important their decisions be transparent, reliable and understandable by humans for better trust and confidence. To this effect, concept-based models such as Concept Bottleneck Models (CBMs) and Self-Explaining Neural Networks (SENN) have been proposed which constrain the latent space of a model to represent high level concepts easily understood by domain experts in the field. Although concept-based models promise a good approach to both increasing explainability and reliability, it is yet to be shown if they demonstrate robustness and output consistent concepts under systematic perturbations to their inputs. To better understand performance of concept-based models on curated malicious samples, in this paper, we aim to study their robustness to adversarial perturbations, which are also known as the imperceptible changes to the input data that are crafted by an attacker to fool a well-learned concept-based model. Specifically, we first propose and analyze different malicious attacks to evaluate the security vulnerability of concept based models. Subsequently, we propose a potential general adversarial training-based defense mechanism to increase robustness of these systems to the proposed malicious attacks. Extensive experiments on one synthetic and two real-world datasets demonstrate the effectiveness of the proposed attacks and the defense approach.