mlsanityData to 7 Oct 2026

A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation

arXiv 2306.07304 ↗First posted 11 Jun 2023cs.LG cs.AIcited by 108

Not on the map. Why?

Abstract (arXiv)

In recent years, concept-based approaches have emerged as some of the most promising explainability methods to help us interpret the decisions of Artificial Neural Networks (ANNs). These methods seek to discover intelligible visual 'concepts' buried within the complex patterns of ANN activations in two key steps: (1) concept extraction followed by (2) importance estimation. While these two steps are shared across methods, they all differ in their specific implementations. Here, we introduce a unifying theoretical framework that comprehensively defines and clarifies these two steps. This framework offers several advantages as it allows us: (i) to propose new evaluation metrics for comparing different concept extraction approaches; (ii) to leverage modern attribution methods and evaluation metrics to extend and systematically evaluate state-of-the-art concept-based approaches and importance estimation techniques; (iii) to derive theoretical guarantees regarding the optimality of such methods. We further leverage our framework to try to tackle a crucial question in explainability: how to efficiently identify clusters of data points that are classified based on a similar shared strategy. To illustrate these findings and to highlight the main strategies of a model, we introduce a visual representation called the strategic cluster graph. Finally, we present https://serre-lab.github.io/Lens, a dedicated website that offers a complete compilation of these visualizations for all classes of the ImageNet dataset.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). Builds on 48 of its 66 references that are in the map. First posted on arXiv 11 Jun 2023.

Builds on

48 of 66 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  3. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  4. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  5. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  6. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  7. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  8. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  9. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  10. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  11. 2018Sanity Checks for Saliency Mapscited by 1,149
  12. 2017SmoothGrad: removing noise by adding noisecited by 1,097
36 more
  1. 2022Toy Models of Superpositioncited by 928
  2. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
  3. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
  4. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  5. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  6. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  7. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 467
  8. 2017Interpretation of Neural Networks is Fragilecited by 443
  9. 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 439
  10. 2017The (Un)reliability of saliency methodscited by 331
  11. 2013k-Sparse Autoencoderscited by 209
  12. 2020Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?cited by 189
  13. 2020Evaluating and Aggregating Feature-based Model Explanationscited by 185
  14. 2022CRAFT: Concept Recursive Activation FacTorization for Explainabilitycited by 177
  15. 2020Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AIcited by 166
  16. 2020Reliable Post hoc Explanations: Modeling Uncertainty in Explainabilitycited by 125
  17. 2020Invertible Concept-based Explanations for CNN Models with Non-negative Concept Activation Vectorscited by 119
  18. 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
  19. 2021What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methodscited by 98
  20. 2019When Explanations Lie: Why Many Modified BP Attributions Failcited by 92
  21. 2021The effectiveness of feature attribution methods and its correlation with automatic evaluation scorescited by 87
  22. 2021Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysiscited by 65
  23. 2020Evaluations and Methods for Explanation through Robustness Analysiscited by 51
  24. 2014Sparse Modeling for Image and Vision Processingcited by 48
  25. 2023Multi-dimensional concept discovery (MCD): A unifying framework with completeness guaranteescited by 48
  26. 2020How Useful Are the Machine-Generated Interpretations to General Users? A Human Evaluation on Guessing the Incorrectly Predicted Labelscited by 39
  27. 2022Don't Lie to Me! Robust and Efficient Explainability with Verified Perturbation Analysiscited by 38
  28. 2022Making Sense of Dependence: Efficient Black-box Explanations Using Dependence Measurecited by 33
  29. 2021On Baselines for Local Feature Attributionscited by 29
  30. 2022Towards Better Understanding Attribution Methodscited by 28
  31. 2022Xplique: A Deep Learning Explainability Toolboxcited by 24
  32. 2022Listen to Interpret: Post-hoc Interpretability for Audio Networks with NMFcited by 23
  33. 2019Computing Linear Restrictions of Neural Networkscited by 14
  34. 2022Do Users Benefit From Interpretable Vision? A User Study, Baseline, And Datasetcited by 13
  35. 2023On the coalitional decomposition of parameters of interestcited by 7
  36. 2021Initialization for Nonnegative Matrix Factorization: a Comprehensive Reviewcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 29; 2025: 43; 2026: 33.

108 papers.

Sort:

By area:

  1. 2024Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)cited by 116
  2. 2024Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discoverycited by 91
  3. 2025Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Modelscited by 67
  4. 2023On the Foundations of Shortcut Learningcited by 64
  5. 2025Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignmentcited by 62
  6. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  7. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 45
  8. 2025From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuitcited by 45
  9. 2024A Concept-Based Explainability Framework for Large Multimodal Modelscited by 39
  10. 2025Interpreting the linear structure of vision-language model embedding spacescited by 32
  11. 2024PURE: Turning Polysemantic Neurons Into Pure Features by Identifying Relevant Circuitscited by 29
  12. 2025Mechanistic understanding and validation of large AI models with SemanticLenscited by 29
38 more
  1. 2026Do Sparse Autoencoders Capture Concept Manifolds?cited by 25
  2. 2024Concept-skill Transferability-based Data Selection for Large Vision-Language Modelscited by 24
  3. 2023Interpretability is in the Mind of the Beholder: A Causal Framework for Human-interpretable Representation Learningcited by 22
  4. 2024Understanding Video Transformers via Universal Concept Discoverycited by 22
  5. 2023Understanding the (Extra-)Ordinary: Validating Deep Model Decisions with Prototypical Concept-based Explanationscited by 21
  6. 2024Concept Bottleneck Models Without Predefined Conceptscited by 17
  7. 2024Concept-Based Explanations in Computer Vision: Where Are We and Where Could We Go?cited by 17
  8. 2025Block-Recurrent Dynamics in Vision Transformerscited by 16
  9. 2024Understanding Visual Feature Reliance through the Lens of Complexitycited by 15
  10. 2025Analyzing Finetuning Representation Shift for Multimodal LLMs Steeringcited by 15
  11. 2024WWW: A Unified Framework for Explaining What, Where and Why of Neural Networks by Interpretation of Neuron Conceptscited by 14
  12. 2024Pruning By Explaining Revisited: Optimizing Attribution Methods to Prune CNNs and Transformerscited by 11
  13. 2024Visual-TCAV: Concept-based Attribution and Saliency Maps for Post-hoc Explainability in Image Classificationcited by 11
  14. 2024Beyond Scalars: Concept-Based Alignment Analysis in Vision Transformerscited by 11
  15. 2024Evaluating Readability and Faithfulness of Concept-based Explanationscited by 10
  16. 2025Object Centric Concept Bottleneckscited by 10
  17. 2025ConSim: Measuring Concept-Based Explanations' Effectiveness with Automated Simulatabilitycited by 9
  18. 2025CoE: Chain-of-Explanation via Automatic Visual Concept Circuit Description and Polysemanticity Quantificationcited by 9
  19. 2026Cross-Modal Redundancy and the Geometry of Vision-Language Embeddingscited by 9
  20. 2025FACE: Faithful Automatic Concept Extractioncited by 8
  21. 2024Probabilistic Conceptual Explainers: Trustworthy Conceptual Explanations for Vision Foundation Modelscited by 6
  22. 2025Towards Achieving Concept Completeness for Textual Concept Bottleneck Modelscited by 6
  23. 2025Quantifying Structure in CLIP Embeddings: A Statistical Framework for Concept Interpretationcited by 6
  24. 2024Reactive Model Correction: Mitigating Harm to Task-Relevant Features via Conditional Bias Suppressioncited by 5
  25. 2024Choosing the right basis for interpretability: Psychophysical comparison between neuron-based and dictionary-based representationscited by 5
  26. 2025Representational Similarity via Interpretable Visual Conceptscited by 5
  27. 2025Towards Human-Understandable Multi-Dimensional Concept Discoverycited by 4
  28. 2025From What to How: Attributing CLIP's Latent Components Reveals Unexpected Semantic Reliancecited by 4
  29. 2026Bi-Orthogonal Factor Decomposition for Vision Transformerscited by 4
  30. 2024Explainable Concept Generation through Vision-Language Preference Learning for Understanding Neural Networks' Internal Representationscited by 3
  31. 2025Ensembling Sparse Autoencoderscited by 3
  32. 2025Mechanistic Interpretability with Sparse Autoencoder Neural Operatorscited by 3
  33. 2025A Geometric Unification of Concept Learning with Concept Conescited by 3
  34. 2026Revisiting Anisotropy in Language Transformers: The Geometry of Learning Dynamicscited by 3
  35. 2026Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifoldscited by 3
  36. 2024DISCRET: Synthesizing Faithful Explanations For Treatment Effect Estimationcited by 2
  37. 2024Towards Utilising a Range of Neural Activations for Comprehending Representational Associationscited by 2
  38. 2025Ensuring Medical AI Safety: Interpretability-Driven Detection and Mitigation of Spurious Model Behavior and Associated Datacited by 2

50 most cited of 108 All 108 (JSON)