# CENDRe: Concept Extraction with Natural Domain Representations

arXiv: https://arxiv.org/abs/2607.29621 · HTML: https://mlsanity.com/paper/2607.29621/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 51 references that are in the map.
- First posted on arXiv 31 Jul 2026.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [InceptionTime: Finding AlexNet for Time Series Classification](https://mlsanity.com/paper/1909.04939/) (2019)
- [On Completeness-aware Concept-Based Explanations in Deep Neural Networks](https://mlsanity.com/paper/1910.07969/) (2019)
- [Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAI](https://mlsanity.com/paper/2003.07258/) (2020)
- [Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypes](https://mlsanity.com/paper/2111.15000/) (2021)
- [Concept-based Explainable Artificial Intelligence: A Survey](https://mlsanity.com/paper/2312.12936/) (2023)
- [Explainable AI for Time Series via Virtual Inspection Layers](https://mlsanity.com/paper/2303.06365/) (2023)
- [ECLAD: Extracting Concepts with Local Aggregated Descriptors](https://mlsanity.com/paper/2206.04531/) (2022)
- [Towards Prototype-Based Self-Explainable Graph Neural Network](https://mlsanity.com/paper/2210.01974/) (2022)
- [PACE: Posthoc Architecture-Agnostic Concept Extractor for Explaining CNNs](https://mlsanity.com/paper/2108.13828/) (2021)
- [Concept-based explainability for an EEG transformer model](https://mlsanity.com/paper/2307.12745/) (2023)
- [Time is Not Enough: Time-Frequency based Explanation for Time-Series Black-Box Models](https://mlsanity.com/paper/2408.03636/) (2024)
- [Scale-Preserving Automatic Concept Extraction (SPACE)](https://mlsanity.com/paper/2308.06022/) (2023)
- [Explaining Deep Neural Networks for Bearing Fault Detection with Vibration Concepts](https://mlsanity.com/paper/2310.11450/) (2023)
- [Concept Extraction for Time Series with ECLAD-ts](https://mlsanity.com/paper/2504.05024/) (2025)
- [From Confusion to Clarity: ProtoScore -- A Framework for Evaluating Prototype-Based XAI](https://mlsanity.com/paper/2511.08361/) (2025)

## Abstract (arXiv)

Convolutional neural networks (CNNs) are widely used for time-series classification, but their deployment in critical domains requires understanding the temporal and spectral patterns that drive their predictions. Concept extraction (CE) methods identify such patterns by analyzing representations within the models' latent space. However, existing time-series CE methods have three limitations: they operate only in the time domain and overlook frequency features, predefine the number of concepts, and produce localizations misaligned with the regions the model uses. We address these limitations by proposing CENDRe, a concept extraction method for CNNs. It first discovers concepts by clustering per-timestep latent representations in two stages, where silhouette-guided aggregation selects the number of concepts automatically. Then, it localizes each concept through gradients of a presence score that contrasts the latent representations with their prototypes, producing masks that concentrate on the regions driving the concept. These gradients, propagated through a differentiable invertible mapping of the input such as a Fourier transform, yield localizations for the same concepts in the frequency domain. Finally, each concept receives a relevance score that quantifies its contribution to each class. On synthetic benchmarks, CENDRe achieves representation correctness comparable to state-of-the-art CE methods and significantly higher importance correctness. On real bearing-fault data, CENDRe extracts the frequency bands driving the model's predictions, located in regions commonly inspected for fault diagnosis, producing evidence to assess the model that time-domain CE methods cannot.