mlsanityData to 7 Oct 2026

Error-margin Analysis for Hidden Neuron Activation Labels

arXiv 2405.09580 ↗First posted 14 May 2024cs.LG cs.AI cs.NEcited by 1

Not on the map. Why?

Abstract (arXiv)

Understanding how high-level concepts are represented within artificial neural networks is a fundamental challenge in the field of artificial intelligence. While existing literature in explainable AI emphasizes the importance of labeling neurons with concepts to understand their functioning, they mostly focus on identifying what stimulus activates a neuron in most cases, this corresponds to the notion of recall in information retrieval. We argue that this is only the first-part of a two-part job, it is imperative to also investigate neuron responses to other stimuli, i.e., their precision. We call this the neuron labels error margin.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 23 of its 34 references that are in the map. First posted on arXiv 14 May 2024.

Builds on

23 of 34 references.

References · 23

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  5. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  6. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  7. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  8. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  9. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  10. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,086
  11. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  12. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 440
11 more
  1. 2018On the Robustness of Interpretability Methodscited by 310
  2. 2023Label-Free Concept Bottleneck Modelscited by 282
  3. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 254
  4. 2016Grad-CAM: Why did you say that?cited by 159
  5. 2022CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networkscited by 144
  6. 2020Invertible Concept-based Explanations for CNN Models with Non-negative Concept Activation Vectorscited by 119
  7. 2017Interpreting Deep Visual Representations via Network Dissectioncited by 103
  8. 2022Concept Activation Regions: A Generalized Framework For Concept-Based Explanationscited by 57
  9. 2021EXplainable Neural-Symbolic Learning (X-NeSyL) methodology to fuse deep learning representations with expert knowledge graphs: the MonuMAI cultural heritage use casecited by 30
  10. 2017Explaining Trained Neural Networks with Semantic Web Technologies: First Stepscited by 13
  11. 2024On the Value of Labeled Data and Symbolic Methods for Hidden Neuron Activation Analysiscited by 6

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024ConceptLens: from Pixels to Understandingcited by 0