mlsanityData to 7 Oct 2026

Latent Concept-based Explanation of NLP Models

arXiv 2404.12545 ↗First posted 18 Apr 2024cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

Interpreting and understanding the predictions made by deep learning models poses a formidable challenge due to their inherently opaque nature. Many previous efforts aimed at explaining these predictions rely on input features, specifically, the words within NLP models. However, such explanations are often less informative due to the discrete nature of these words and their lack of contextual verbosity. To address this limitation, we introduce the Latent Concept Attribution method (LACOAT), which generates explanations for predictions based on latent concepts. Our foundational intuition is that a word can exhibit multiple facets, contingent upon the context in which it is used. Therefore, given a word in context, the latent space derived from our training process reflects a specific facet of that word. LACOAT functions by mapping the representations of salient input words into the training latent space, allowing it to provide latent context-based explanations of the prediction.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 35 of its 65 references that are in the map. First posted on arXiv 18 Apr 2024.

Builds on

35 of 65 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  7. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  8. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  9. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  10. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  11. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  12. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
23 more
  1. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  2. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  3. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  4. 2019Revealing the Dark Secrets of BERTcited by 300
  5. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 255
  6. 2019Visualizing and Measuring the Geometry of BERTcited by 215
  7. 2023AutoAgents: A Framework for Automatic Agent Generationcited by 211
  8. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 155
  9. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  10. 2021Guided Integrated Gradients: An Adaptive Path Method for Removing Noisecited by 102
  11. 2023Explaining black box text modules in natural language with language modelscited by 61
  12. 2022Discovering Latent Concepts Learned in BERTcited by 56
  13. 2022CEBaB: Estimating the Causal Effects of Real-World Concepts on NLP Model Behaviorcited by 54
  14. 2014Extraction of Salient Sentences from Labelled Documentscited by 50
  15. 2021SelfExplain: A Self-Explaining Architecture for Neural Text Classifierscited by 44
  16. 2020Explaining and Improving Model Behavior with k Nearest Neighbor Representationscited by 27
  17. 2022On the Transformation of Latent Space in Fine-Tuned NLP Modelscited by 19
  18. 2023COCKATIEL: COntinuous Concept ranKed ATtribution with Interpretable ELements for explaining neural net classifiers on NLP taskscited by 16
  19. 2023Can LLMs facilitate interpretation of pre-trained language models?cited by 15
  20. 2023Incorporating Attribution Importance for Improving Faithfulness Metricscited by 14
  21. 2023NeuroX Library for Neuron Analysis of Deep NLP Modelscited by 11
  22. 2023Scaling up Discovery of Latent Concepts in Deep NLP Modelscited by 9
  23. 2023Explaining Language Models' Predictions with High-Impact Conceptscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 8; 2026: 2.

13 papers.

Sort:

  1. 2024Reference-Guided Verdict: LLMs-as-Judges in Automatic Evaluation of Free-Form QAcited by 25
  2. 2025Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Frameworkcited by 9
  3. 2025CLEV: LLM-Based Evaluation Through Lightweight Efficient Voting for Free-Form Question-Answeringcited by 6
  4. 2025B-cos LM: Efficiently Transforming Pre-trained Language Models for Improved Explainabilitycited by 4
  5. 2025Interpreting Language Models Through Concept Descriptions: A Surveycited by 3
  6. 2024DLBacktrace: A Model Agnostic Explainability for any Deep Learning Modelscited by 2
  7. 2025SAGE: A Search-AuGmented Evaluation of Large Language Models on Free-Form QAcited by 2
  8. 2024Beyond Attribution: Unified Concept-Level Explanationscited by 1
  9. 2025Cross-Layer Discrete Concept Discovery for Interpreting Language Modelscited by 1
  10. 2025Analyzing Latent Concepts in Code Language Modelscited by 1
  11. 2026LIBERTy: A Causal Framework for Benchmarking Concept-Based Explanations of LLMs with Structural Counterfactualscited by 1
  12. 2025Bridging Fairness and Explainability: Can Input-Based Explanations Promote Fairness in Hate Speech Detection?cited by 0
1 more
  1. 2026Vector Quantized Latent Concepts: A Scalable Alternative to Clustering-Based Concept Discoverycited by 0