# Intrinsic Probing through Dimension Selection

arXiv: https://arxiv.org/abs/2010.02812 · HTML: https://mlsanity.com/paper/2010.02812/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Oct 2020.

## Built on by (most cited first, 30 of 45)

- [Finding Neurons in a Haystack: Case Studies with Sparse Probing](https://mlsanity.com/paper/2305.01610/) (2023)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [Neuron-level Interpretation of Deep NLP Models: A Survey](https://mlsanity.com/paper/2108.13138/) (2021)
- [Finding Skill Neurons in Pre-trained Transformer-based Language Models](https://mlsanity.com/paper/2211.07349/) (2022)
- [On the Pitfalls of Analyzing Individual Neurons in Language Models](https://mlsanity.com/paper/2110.07483/) (2021)
- [Probing for the Usage of Grammatical Number](https://mlsanity.com/paper/2204.08831/) (2022)
- [Discovering Latent Concepts Learned in BERT](https://mlsanity.com/paper/2205.07237/) (2022)
- [Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2412.02104/) (2024)
- [Probing the Emergence of Cross-lingual Alignment during LLM Training](https://mlsanity.com/paper/2406.13229/) (2024)
- [A Survey on Mechanistic Interpretability for Multi-Modal Foundation Models](https://mlsanity.com/paper/2502.17516/) (2025)
- [Neuron to Graph: Interpreting Language Model Neurons at Scale](https://mlsanity.com/paper/2305.19911/) (2023)
- [Holmes: A Benchmark to Assess the Linguistic Competence of Language Models](https://mlsanity.com/paper/2404.18923/) (2024)
- [How is BERT surprised? Layerwise detection of linguistic anomalies](https://mlsanity.com/paper/2105.07452/) (2021)
- [A Bayesian Framework for Information-Theoretic Probing](https://mlsanity.com/paper/2109.03853/) (2021)
- [Discovering Salient Neurons in Deep NLP Models](https://mlsanity.com/paper/2206.13288/) (2022)
- [Can LLMs facilitate interpretation of pre-trained language models?](https://mlsanity.com/paper/2305.13386/) (2023)
- [Towards a fuller understanding of neurons with Clustered Compositional Explanations](https://mlsanity.com/paper/2310.18443/) (2023)
- [Same Neurons, Different Languages: Probing Morphosyntax in Multilingual Pre-trained Models](https://mlsanity.com/paper/2205.02023/) (2022)
- [Probing as Quantifying Inductive Bias](https://mlsanity.com/paper/2110.08388/) (2021)
- [NeuroX Library for Neuron Analysis of Deep NLP Models](https://mlsanity.com/paper/2305.17073/) (2023)
- [Evaluating Readability and Faithfulness of Concept-based Explanations](https://mlsanity.com/paper/2404.18533/) (2024)
- [Controlled Evaluation of Syntactic Knowledge in Multilingual Language Models](https://mlsanity.com/paper/2411.07474/) (2024)
- [Evaluating Neuron Interpretation Methods of NLP Models](https://mlsanity.com/paper/2301.12608/) (2023)
- [UniFusion: Vision-Language Model as Unified Encoder in Image Generation](https://mlsanity.com/paper/2510.12789/) (2025)
- [SocioProbe: What, When, and Where Language Models Learn about Sociodemographics](https://mlsanity.com/paper/2211.04281/) (2022)
- [Code-Mixed Probes Show How Pre-Trained Models Generalise On Code-Switched Text](https://mlsanity.com/paper/2403.04872/) (2024)
- [John praised Mary because he? Implicit Causality Bias and Its Interaction with Explicit Cues in LMs](https://mlsanity.com/paper/2106.01060/) (2021)
- [A Latent-Variable Model for Intrinsic Probing](https://mlsanity.com/paper/2201.08214/) (2022)
- [A Joint Matrix Factorization Analysis of Multilingual Representations](https://mlsanity.com/paper/2310.15513/) (2023)
- [Multilingual Nonce Dependency Treebanks: Understanding how Language Models represent and process syntactic structure](https://mlsanity.com/paper/2311.07497/) (2023)

## Abstract (arXiv)

Most modern NLP systems make use of pre-trained contextual representations that attain astonishingly high performance on a variety of tasks. Such high performance should not be possible unless some form of linguistic structure inheres in these representations, and a wealth of research has sprung up on probing for it. In this paper, we draw a distinction between intrinsic probing, which examines how linguistic information is structured within a representation, and the extrinsic probing popular in prior work, which only argues for the presence of such information by showing that it can be successfully extracted. To enable intrinsic probing, we propose a novel framework based on a decomposable multivariate Gaussian probe that allows us to determine whether the linguistic information in word embeddings is dispersed or focal. We then probe fastText and BERT for various morphosyntactic attributes across 36 languages. We find that most attributes are reliably encoded by only a few neurons, with fastText concentrating its linguistic structure more than BERT.