mlsanityData to 7 Oct 2026

Intrinsic Probing through Dimension Selection

arXiv 2010.02812 ↗First posted 6 Oct 2020cs.CLcited by 45

Not on the map. Why?

Abstract (arXiv)

Most modern NLP systems make use of pre-trained contextual representations that attain astonishingly high performance on a variety of tasks. Such high performance should not be possible unless some form of linguistic structure inheres in these representations, and a wealth of research has sprung up on probing for it. In this paper, we draw a distinction between intrinsic probing, which examines how linguistic information is structured within a representation, and the extrinsic probing popular in prior work, which only argues for the presence of such information by showing that it can be successfully extracted. To enable intrinsic probing, we propose a novel framework based on a decomposable multivariate Gaussian probe that allows us to determine whether the linguistic information in word embeddings is dispersed or focal. We then probe fastText and BERT for various morphosyntactic attributes across 36 languages. We find that most attributes are reliably encoded by only a few neurons, with fastText concentrating its linguistic structure more than BERT.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 10; 2023: 11; 2024: 8; 2025: 5; 2026: 1.

45 papers.

Sort:

By area:

  1. 2023Explainability for Large Language Models: A Surveycited by 331
  2. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  3. 2021Neuron-level Interpretation of Deep NLP Models: A Surveycited by 81
  4. 2022Finding Skill Neurons in Pre-trained Transformer-based Language Modelscited by 65
  5. 2021On the Pitfalls of Analyzing Individual Neurons in Language Modelscited by 61
  6. 2022Probing for the Usage of Grammatical Numbercited by 57
  7. 2022Discovering Latent Concepts Learned in BERTcited by 56
  8. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  9. 2024Probing the Emergence of Cross-lingual Alignment during LLM Trainingcited by 38
  10. 2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
  11. 2023Neuron to Graph: Interpreting Language Model Neurons at Scalecited by 26
  12. 2024Holmes: A Benchmark to Assess the Linguistic Competence of Language Modelscited by 26
33 more
  1. 2021How is BERT surprised? Layerwise detection of linguistic anomaliescited by 24
  2. 2021A Bayesian Framework for Information-Theoretic Probingcited by 22
  3. 2022Discovering Salient Neurons in Deep NLP Modelscited by 21
  4. 2023Can LLMs facilitate interpretation of pre-trained language models?cited by 15
  5. 2023Towards a fuller understanding of neurons with Clustered Compositional Explanationscited by 15
  6. 2022Same Neurons, Different Languages: Probing Morphosyntax in Multilingual Pre-trained Modelscited by 14
  7. 2021Probing as Quantifying Inductive Biascited by 11
  8. 2023NeuroX Library for Neuron Analysis of Deep NLP Modelscited by 11
  9. 2024Evaluating Readability and Faithfulness of Concept-based Explanationscited by 10
  10. 2024Controlled Evaluation of Syntactic Knowledge in Multilingual Language Modelscited by 10
  11. 2023Evaluating Neuron Interpretation Methods of NLP Modelscited by 8
  12. 2022SocioProbe: What, When, and Where Language Models Learn about Sociodemographicscited by 7
  13. 2024Code-Mixed Probes Show How Pre-Trained Models Generalise On Code-Switched Textcited by 7
  14. 2025UniFusion: Vision-Language Model as Unified Encoder in Image Generationcited by 7
  15. 2021John praised Mary because he? Implicit Causality Bias and Its Interaction with Explicit Cues in LMscited by 5
  16. 2022A Latent-Variable Model for Intrinsic Probingcited by 5
  17. 2023A Joint Matrix Factorization Analysis of Multilingual Representationscited by 5
  18. 2023Multilingual Nonce Dependency Treebanks: Understanding how Language Models represent and process syntactic structurecited by 5
  19. 2021Fine-grained Interpretation and Causation Analysis in Deep NLP Modelscited by 4
  20. 2022On the data requirements of probingcited by 4
  21. 2022IDANI: Inference-time Domain Adaptation via Neuron-level Interventionscited by 4
  22. 2023N2G: A Scalable Approach for Quantifying Interpretable Neuron Representations in Large Language Modelscited by 4
  23. 2022Probing with Noise: Unpicking the Warp and Weft of Embeddingscited by 3
  24. 2023Idioms, Probing and Dangerous Things: Towards Structural Probing for Idiomaticity in Vector Spacecited by 3
  25. 2025Disentangling Linguistic Features with Dimension-Wise Analysis of Vector Embeddingscited by 2
  26. 2021Do RNN States Encode Abstract Phonological Processes?cited by 1
  27. 2021Unsupervised Representation Disentanglement of Text: An Evaluation on Synthetic Datasetscited by 1
  28. 2024Towards Generating Informative Textual Description for Neurons in Language Modelscited by 1
  29. 2024Linguistically Grounded Analysis of Language Models using Shapley Head Valuescited by 1
  30. 2025ParaUni: Enhance Generation in Unified Multimodal Model with Reinforcement-driven Hierarchical Parallel Information Interactioncited by 1
  31. 2026The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Modelscited by 1
  32. 2021Shape of Elephant: Study of Macro Properties of Word Embeddings Spacescited by 0
  33. 2025Example-Based Concept Analysis Framework for Deep Weather Forecast Modelscited by 0