# On The Variability of Concept Activation Vectors

arXiv: https://arxiv.org/abs/2509.24058 · HTML: https://mlsanity.com/paper/2509.24058/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 35 references that are in the map.
- First posted on arXiv 28 Sep 2025.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [On the Robustness of Interpretability Methods](https://mlsanity.com/paper/1806.08049/) (2018)
- [Concept Whitening for Interpretable Image Recognition](https://mlsanity.com/paper/2002.01650/) (2020)
- [Invertible Concept-based Explanations for CNN Models with Non-negative Concept Activation Vectors](https://mlsanity.com/paper/2006.15417/) (2020)
- [Concept-based Explainable Artificial Intelligence: A Survey](https://mlsanity.com/paper/2312.12936/) (2023)
- [Statistical stability indices for LIME: obtaining reliable explanations for Machine Learning models](https://mlsanity.com/paper/2001.11757/) (2020)
- [What does LIME really see in images?](https://mlsanity.com/paper/2102.06307/) (2021)
- [Concept Gradient: Concept-based Interpretation Without Linear Assumption](https://mlsanity.com/paper/2208.14966/) (2022)
- [From Hope to Safety: Unlearning Biases of Deep Models via Gradient Penalization in Latent Space](https://mlsanity.com/paper/2308.09437/) (2023)
- [Understanding Inter-Concept Relationships in Concept-Based Models](https://mlsanity.com/paper/2405.18217/) (2024)
- [Adversarial TCAV -- Robust and Effective Interpretation of Intermediate Layers in Neural Networks](https://mlsanity.com/paper/2002.03549/) (2020)
- [Spatial-temporal Concept based Explanation of 3D ConvNets](https://mlsanity.com/paper/2206.05275/) (2022)
- [Visual-TCAV: Concept-based Attribution and Saliency Maps for Post-hoc Explainability in Image Classification](https://mlsanity.com/paper/2411.05698/) (2024)
- [Reactive Model Correction: Mitigating Harm to Task-Relevant Features via Conditional Bias Suppression](https://mlsanity.com/paper/2404.09601/) (2024)
- [Asymptotic Inference for Infinitely Imbalanced Logistic Regression](https://mlsanity.com/paper/2204.13231/) (2022)

## Built on by (most cited first, 3 of 3)

- [PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning](https://mlsanity.com/paper/2601.13020/) (2026)
- [A High-Resolution Landscape Dataset for Concept-Based XAI With Application to Species Distribution Models](https://mlsanity.com/paper/2604.13240/) (2026)
- [α-TCAV: A Unified Framework for Testing with Concept Activation Vectors](https://mlsanity.com/paper/2605.15688/) (2026)

## Abstract (arXiv)

One of the most pressing challenges in artificial intelligence is to make models more transparent to their users. Recently, explainable artificial intelligence has come up with numerous method to tackle this challenge. A promising avenue is to use concept-based explanations, that is, high-level concepts instead of plain feature importance score. Among this class of methods, Concept Activation vectors (CAVs), Kim et al. (2018) stands out as one of the main protagonists. One interesting aspect of CAVs is that their computation requires sampling random examples in the train set. Therefore, the actual vectors obtained may vary from user to user depending on the randomness of this sampling. In this paper, we propose a fine-grained theoretical analysis of CAVs construction in order to quantify their variability. Our results, confirmed by experiments on several real-life datasets, point out towards an universal result: the variance of CAVs decreases as 1/N, where N is the number of random examples. Based on this we give practical recommendations for a resource-efficient application of the method.