mlsanityData to 7 Oct 2026

Understanding intermediate layers using linear classifier probes

arXiv 1610.01644 ↗First posted 5 Oct 2016stat.ML cs.LGcited by 1,279

Not on the map. Why?

Cited by 88 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Neural network models have a reputation for being black boxes. We propose to monitor the features at every layer of a model and measure how suitable they are for classification. We use linear classifiers, which we refer to as "probes", trained entirely independently of the model itself. This helps us better understand the roles and dynamics of the intermediate layers. We demonstrate how this can be used to develop a better intuition about models and to diagnose potential problems. We apply this technique to the popular models Inception v3 and Resnet-50. Among other things, we observe experimentally that the linear separability of features increase monotonically along the depth of the model.

Key facts. Cited by 1,279 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Oct 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 61; 2022: 71; 2023: 99; 2024: 170; 2025: 281; 2026: 597.

1,279 papers.

Sort:

By area:

  1. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  2. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  3. 2023Steering Language Models With Activation Engineeringcited by 824
  4. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  5. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  6. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  7. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  8. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  9. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 447
  10. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  11. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  12. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
38 more
  1. 2023Language Models Represent Space and Timecited by 306
  2. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 293
  3. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 246
  4. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  5. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 225
  6. 2021Revisiting Model Stitching to Compare Neural Representationscited by 203
  7. 2024Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomographycited by 192
  8. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  9. 2021Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Littlecited by 176
  10. 2021Poisoning and Backdooring Contrastive Learningcited by 149
  11. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 148
  12. 2023The Hydra Effect: Emergent Self-repair in Language Model Computationscited by 145
  13. 2021Deep Learning Through the Lens of Example Difficultycited by 139
  14. 2023Looped Transformers are Better at Learning Learning Algorithmscited by 135
  15. 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 133
  16. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  17. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  18. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  19. 2021Acquisition of Chess Knowledge in AlphaZerocited by 120
  20. 2024FeatUp: A Model-Agnostic Framework for Features at Any Resolutioncited by 114
  21. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 109
  22. 2021Neural Networks as Kernel Learners: The Silent Alignment Effectcited by 106
  23. 2023Uncovering mesa-optimization algorithms in Transformerscited by 94
  24. 2021Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?cited by 91
  25. 2024TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Spacecited by 90
  26. 2021Video Summarization Using Deep Neural Networks: A Surveycited by 85
  27. 2021Ensembling Off-the-shelf Models for GAN Trainingcited by 84
  28. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  29. 2022Reliability of CKA as a Similarity Measure in Deep Learningcited by 82
  30. 2024Truth is Universal: Robust Detection of Lies in LLMscited by 78
  31. 2023Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblockscited by 77
  32. 2023SODA: Bottleneck Diffusion Models for Representation Learningcited by 76
  33. 2022Human alignment of neural network representationscited by 74
  34. 2024Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categoriescited by 74
  35. 2023How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representationscited by 72
  36. 2021Chess as a Testbed for Language Model State Trackingcited by 69
  37. 2023Do Androids Know They're Only Dreaming of Electric Sheep?cited by 67
  38. 2021First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERTcited by 64

50 most cited of 1,279 All 1,279 (JSON)