# Concept Gradient: Concept-based Interpretation Without Linear Assumption

arXiv: https://arxiv.org/abs/2208.14966 · HTML: https://mlsanity.com/paper/2208.14966/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 54 references that are in the map.
- First posted on arXiv 31 Aug 2022.

## Builds on (most cited first, 30 of 34)

- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Estimating Training Data Influence by Tracing Gradient Descent](https://mlsanity.com/paper/2002.08484/) (2020)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [Counterfactual Visual Explanations](https://mlsanity.com/paper/1904.07451/) (2019)
- [On the (In)fidelity and Sensitivity for Explanations](https://mlsanity.com/paper/1901.09392/) (2019)
- [Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives](https://mlsanity.com/paper/1802.07623/) (2018)
- [Real Time Image Saliency for Black Box Classifiers](https://mlsanity.com/paper/1705.07857/) (2017)
- [On Completeness-aware Concept-Based Explanations in Deep Neural Networks](https://mlsanity.com/paper/1910.07969/) (2019)
- [FACE: Feasible and Actionable Counterfactual Explanations](https://mlsanity.com/paper/1909.09369/) (2019)
- [Concept Whitening for Interpretable Image Recognition](https://mlsanity.com/paper/2002.01650/) (2020)
- [Representer Point Selection for Explaining Deep Neural Networks](https://mlsanity.com/paper/1811.09720/) (2018)
- [Visualizing Deep Neural Network Decisions: Prediction Difference Analysis](https://mlsanity.com/paper/1702.04595/) (2017)
- [Explaining Classifiers with Causal Concept Effect (CaCE)](https://mlsanity.com/paper/1907.07165/) (2019)
- [Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems](https://mlsanity.com/paper/1907.09615/) (2019)
- [Grounding Visual Explanations](https://mlsanity.com/paper/1807.09685/) (2018)
- [Falling Rule Lists](https://mlsanity.com/paper/1411.5899/) (2014)
- [Evaluations and Methods for Explanation through Robustness Analysis](https://mlsanity.com/paper/2006.00442/) (2020)
- [Interpreting Black Box Predictions using Fisher Kernels](https://mlsanity.com/paper/1810.10118/) (2018)
- [DISSECT: Disentangled Simultaneous Explanations via Concept Traversals](https://mlsanity.com/paper/2105.15164/) (2021)
- [Contrastive Explanations with Local Foil Trees](https://mlsanity.com/paper/1806.07470/) (2018)
- [Debiasing Concept-based Explanations with Causal Analysis](https://mlsanity.com/paper/2007.11500/) (2020)

## Built on by (most cited first, 20 of 20)

- [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022)
- [Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Era](https://mlsanity.com/paper/2403.08946/) (2024)
- [Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero](https://mlsanity.com/paper/2310.16410/) (2023)
- [LG-CAV: Train Any Concept Activation Vector with Language Guidance](https://mlsanity.com/paper/2410.10308/) (2024)
- [A survey on Concept-based Approaches For Model Improvement](https://mlsanity.com/paper/2403.14566/) (2024)
- [Large Language Models are Interpretable Learners](https://mlsanity.com/paper/2406.17224/) (2024)
- [Selective Concept Models: Permitting Stakeholder Customisation at Test-Time](https://mlsanity.com/paper/2306.08424/) (2023)
- [Concept-based Analysis of Neural Networks via Vision-Language Models](https://mlsanity.com/paper/2403.19837/) (2024)
- [Debugging and Runtime Analysis of Neural Networks with VLMs (A Case Study)](https://mlsanity.com/paper/2503.17416/) (2025)
- [On The Variability of Concept Activation Vectors](https://mlsanity.com/paper/2509.24058/) (2025)
- [Nonparametric Identification of Latent Concepts](https://mlsanity.com/paper/2510.00136/) (2025)
- [Explaining Deep Neural Networks for Bearing Fault Detection with Vibration Concepts](https://mlsanity.com/paper/2310.11450/) (2023)
- [Adaptive Diagnostic Reasoning Framework for Pathology with Multimodal Large Language Models](https://mlsanity.com/paper/2511.12008/) (2025)
- [LatentExplainer: Explaining Latent Representations in Deep Generative Models with Multimodal Large Language Models](https://mlsanity.com/paper/2406.14862/) (2024)
- [Concept-Based Mechanistic Interpretability Using Structured Knowledge Graphs](https://mlsanity.com/paper/2507.05810/) (2025)
- [Concept-Based Interpretability for Toxicity Detection](https://mlsanity.com/paper/2511.16689/) (2025)
- [An Axiomatic Approach to Model-Agnostic Concept Explanations](https://mlsanity.com/paper/2401.06890/) (2024)
- [Concept Learning for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2507.20143/) (2025)
- [Probing the Probes: Methods and Metrics for Concept Alignment](https://mlsanity.com/paper/2511.04312/) (2025)
- [Improving Adversarial Robustness of Attribution via Implicit Regularization](https://mlsanity.com/paper/2605.29983/) (2026)

## Abstract (arXiv)

Concept-based interpretations of black-box models are often more intuitive for humans to understand. The most widely adopted approach for concept-based interpretation is Concept Activation Vector (CAV). CAV relies on learning a linear relation between some latent representation of a given model and concepts. The linear separability is usually implicitly assumed but does not hold true in general. In this work, we started from the original intent of concept-based interpretation and proposed Concept Gradient (CG), extending concept-based interpretation beyond linear concept functions. We showed that for a general (potentially non-linear) concept, we can mathematically evaluate how a small change of concept affecting the model's prediction, which leads to an extension of gradient-based interpretation to the concept space. We demonstrated empirically that CG outperforms CAV in both toy examples and real world datasets.