mlsanityData to 7 Oct 2026

Probing artificial neural networks: insights from neuroscience

arXiv 2104.08197 ↗First posted 16 Apr 2021cs.LG cs.CLcited by 17

Not on the map. Why?

Abstract (arXiv)

A major challenge in both neuroscience and machine learning is the development of useful tools for understanding complex information processing systems. One such tool is probes, i.e., supervised models that relate features of interest to activation patterns arising in biological or artificial neural networks. Neuroscience has paved the way in using such models through numerous studies conducted in recent decades. In this work, we draw insights from neuroscience to help guide probing research in machine learning. We highlight two important design choices for probes - direction and expressivity - and relate these choices to research goals. We argue that specific research goals play a paramount role when designing a probe and encourage future probing studies to be explicit in stating these goals.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 14 of its 42 references that are in the map. First posted on arXiv 16 Apr 2021.

Builds on

14 of 42 references.

References · 14

  1. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  2. 2019Designing and Interpreting Probes with Control Taskscited by 608
  3. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 382
  4. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 230
  5. 2016Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Taskscited by 216
  6. 2020Information-Theoretic Probing for Linguistic Structurecited by 204
  7. 2020Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?cited by 144
  8. 2018Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Informationcited by 97
  9. 2020Do Language Embeddings Capture Scales?cited by 63
  10. 2020Pareto Probing: Trading Off Accuracy for Complexitycited by 55
  11. 2021Neural population geometry: An approach for understanding biological and artificial neural networkscited by 52
  12. 2020Do Neural Language Models Show Preferences for Syntactic Formalisms?cited by 30
2 more
  1. 2018Interpreting Encoding and Decoding Modelscited by 21
  2. 2016From phonemes to images: levels of representation in a recurrent neural model of visually-grounded language learningcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 1; 2023: 6; 2024: 3; 2025: 3; 2026: 2.

17 papers.

Sort:

By area:

  1. 2021Conditional probing: measuring usable information beyond a baselinecited by 61
  2. 2023SkinCon: A skin disease dataset densely annotated by domain experts for fine-grained model debugging and analysiscited by 44
  3. 2023Dissociating language and thought in large language modelscited by 27
  4. 2024Language Models as Models of Languagecited by 24
  5. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 18
  6. 2021Neural Population Geometry Reveals the Role of Stochasticity in Robust Perceptioncited by 12
  7. 2023Probing Graph Representationscited by 10
  8. 2026From Weights to Activations: Is Steering the Next Frontier of Adaptation?cited by 8
  9. 2022Trustworthy Social Bias Measurementcited by 7
  10. 2024Multilevel Interpretability Of Artificial Neural Networks: Leveraging Framework And Methods From Neurosciencecited by 5
  11. 2024Vision-Language Models Align with Human Neural Representations in Concept Processingcited by 2
  12. 2025Prophecy: Inferring Formal Properties from Neuron Activationscited by 2
5 more
  1. 2023Language Cognition and Language Computation -- Human and Machine Language Understandingcited by 1
  2. 2023Can We Use Probing to Better Understand Fine-tuning and Knowledge Distillation of the BERT NLU?cited by 1
  3. 2023Rethinking the Construction of Effective Metrics for Understanding the Mechanisms of Pretrained Language Modelscited by 0
  4. 2025RouterKT: Mixture-of-Experts for Knowledge Tracingcited by 0
  5. 2026Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probecited by 0