mlsanityData to 14 Oct 2026

Probing Classifiers: Promises, Shortcomings, and Advances

arXiv 2102.12452 ↗First posted 24 Feb 2021cs.CLcited by 876

Not on the map. Why?

Cited by 124 in Mechanistic interpretability, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Probing classifiers have emerged as one of the prominent methodologies for interpreting and analyzing deep neural network models of natural language processing. The basic idea is simple -- a classifier is trained to predict some linguistic property from a model's representations -- and has been used to examine a wide variety of models and properties. However, recent studies have demonstrated various methodological limitations of this approach. This article critically reviews the probing classifiers framework, highlighting their promises, shortcomings, and advances.

Key facts. Cited by 876 papers in the map (data through 14 Oct 2026). Builds on 36 of its 62 references that are in the map. First posted on arXiv 24 Feb 2021.

Builds on

36 of 62 references.

References · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  3. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  4. 2020A Primer in BERTology: What we know about how BERT workscited by 815
  5. 2019Designing and Interpreting Probes with Control Taskscited by 640
  6. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 457
  7. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 438
  8. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 383
  9. 2018Analysis Methods in Neural Language Processing: A Surveycited by 277
  10. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 232
  11. 2016Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Taskscited by 216
  12. 2020Information-Theoretic Probing for Linguistic Structurecited by 207
24 more
  1. 2020A Survey of the State of Explainable AI for Natural Language Processingcited by 206
  2. 2017What do Neural Machine Translation Models Learn about Morphology?cited by 166
  3. 2020Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?cited by 151
  4. 2020CausaLM: Causal Model Explanation Through Counterfactual Language Modelscited by 117
  5. 2018Identifying and Controlling Important Neurons in Neural Machine Translationcited by 111
  6. 2020Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERTcited by 109
  7. 2020When Do You Need Billions of Words of Pretraining Data?cited by 104
  8. 2018Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Informationcited by 97
  9. 2019The emergence of number and syntax units in LSTM language modelscited by 94
  10. 2019Do Attention Heads in BERT Track Syntactic Dependencies?cited by 88
  11. 2018Evaluating Layers of Representation in Neural Machine Translation on Part-of-Speech and Semantic Tagging Taskscited by 56
  12. 2020Pareto Probing: Trading Off Accuracy for Complexitycited by 55
  13. 2020A Tale of a Probe and a Parsercited by 40
  14. 2019On the Linguistic Representational Power of Neural Machine Translation Modelscited by 34
  15. 2020Asking without Telling: Exploring Latent Ontologies in Contextual Representationscited by 34
  16. 2019Correlating neural and symbolic representations of languagecited by 33
  17. 2017Analyzing Hidden Representations in End-to-End Automatic Speech Recognition Systemscited by 32
  18. 2020DeFormer: Decomposing Pre-trained Transformers for Faster Question Answeringcited by 29
  19. 2020Investigating Transferability in Pretrained Language Modelscited by 28
  20. 2019From Balustrades to Pierre Vinken: Looking for Syntax in Transformer Self-Attentionscited by 22
  21. 2020An information theoretic view on selecting linguistic probescited by 19
  22. 2020Picking BERT's Brain: Probing for Linguistic Dependencies in Contextualized Embeddings Using Representational Similarity Analysiscited by 18
  23. 2020Analyzing analytical methods: The case of phonology in neural models of spoken languagecited by 16
  24. 2018Hierarchical Multitask Learning for CTC-based Speech Recognitioncited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 40; 2023: 85; 2024: 141; 2025: 179; 2026: 426.

876 papers.

Sort:

By area:

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  2. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  3. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  4. 2023Large Language Modelscited by 780
  5. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  6. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  7. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  8. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  9. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
  10. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  11. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 404
  12. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
38 more
  1. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 334
  2. 2023Explainability for Large Language Models: A Surveycited by 333
  3. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
  4. 2023Language Models Represent Space and Timecited by 316
  5. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 259
  6. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Taskscited by 246
  7. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 244
  8. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
  9. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 231
  10. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 202
  11. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 200
  12. 2022CREPE: Can Vision-Language Foundation Models Reason Compositionally?cited by 181
  13. 2024Eight Methods to Evaluate Robust Unlearning in LLMscited by 163
  14. 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verificationcited by 159
  15. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 158
  16. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 156
  17. 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 146
  18. 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 131
  19. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 129
  20. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
  21. 2021Acquisition of Chess Knowledge in AlphaZerocited by 123
  22. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 121
  23. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 118
  24. 2023Tracr: Compiled Transformers as a Laboratory for Interpretabilitycited by 104
  25. 2024Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristicscited by 97
  26. 2024Representation Noising: A Defence Mechanism Against Harmful Finetuningcited by 96
  27. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  28. 2024TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Spacecited by 91
  29. 2024In-Context Language Learning: Architectures and Algorithmscited by 89
  30. 2023How Do Transformers Learn Topic Structure: Towards a Mechanistic Understandingcited by 86
  31. 2024LLM Internal States Reveal Hallucination Risk Faced With a Querycited by 82
  32. 2024Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queriescited by 78
  33. 2024Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categoriescited by 77
  34. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 76
  35. 2023What Do Self-Supervised Speech Models Know About Words?cited by 64
  36. 2024How Large Language Models Encode Context Knowledge? A Layer-Wise Probing Studycited by 64
  37. 2024LLM Circuit Analyses Are Consistent Across Training and Scalecited by 63
  38. 2023Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?cited by 62

50 most cited of 876 All 876 (JSON)