mlsanityData to 14 Oct 2026

A Bayesian Framework for Information-Theoretic Probing

arXiv 2109.03853 ↗First posted 8 Sep 2021cs.CL cs.IT math.ITcited by 22

Not on the map. Why?

Abstract (arXiv)

Pimentel et al. (2020) recently analysed probing from an information-theoretic perspective. They argue that probing should be seen as approximating a mutual information. This led to the rather unintuitive conclusion that representations encode exactly the same information about a target task as the original sentences. The mutual information, however, assumes the true probability distribution of a pair of random variables is known, leading to unintuitive results in settings where it is not. This paper proposes a new framework to measure what we term Bayesian mutual information, which analyses information from the perspective of Bayesian agents -- allowing for more intuitive findings in scenarios with finite data. For instance, under Bayesian MI we have that data can add information, processing can help, and information can hurt, which makes it more intuitive for machine learning applications. Finally, we apply our framework to probing where we believe Bayesian mutual information naturally operationalises ease of extraction by explicitly limiting the available background knowledge to solve a task.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 29 of its 60 references that are in the map. First posted on arXiv 8 Sep 2021.

Builds on

29 of 60 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  4. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  5. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  6. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  7. 2019Designing and Interpreting Probes with Control Taskscited by 640
  8. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 438
  9. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 432
  10. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 232
  11. 2016Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Taskscited by 216
  12. 2020Information-Theoretic Probing for Linguistic Structurecited by 207
17 more
  1. 2018Formal Limitations on the Measurement of Mutual Informationcited by 189
  2. 2020Compositional Explanations of Neuronscited by 172
  3. 2020A Theory of Usable Information Under Computational Constraintscited by 162
  4. 2019oLMpics -- On what Language Model Pre-training Capturescited by 161
  5. 2020Analyzing Individual Neurons in Pre-trained Language Modelscited by 97
  6. 2021Probing Across Time: What Does RoBERTa Know and When?cited by 73
  7. 2018The Description Length of Deep Learning Modelscited by 72
  8. 2018Evaluating Layers of Representation in Neural Machine Translation on Part-of-Speech and Semantic Tagging Taskscited by 56
  9. 2020Pareto Probing: Trading Off Accuracy for Complexitycited by 55
  10. 2021What Context Features Can Transformer Language Models Use?cited by 50
  11. 2021Measuring and Increasing Context Usage in Context-Aware Machine Translationcited by 48
  12. 2020Intrinsic Probing through Dimension Selectioncited by 45
  13. 2020A Tale of a Probe and a Parsercited by 40
  14. 2020It's Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Informationcited by 26
  15. 2019Meaning to Form: Measuring Systematicity as Informationcited by 16
  16. 2020Evaluating representations by the complexity of learning low-loss predictorscited by 16
  17. 2019Dominantly Truthful Multi-task Peer Prediction with a Constant Number of Taskscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 7; 2023: 7; 2024: 2; 2025: 0; 2026: 2.

22 papers.

Sort:

By area:

  1. 2021Understanding Dataset Difficulty with V-Usable Informationcited by 291
  2. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Taskscited by 246
  3. 2021Acquisition of Chess Knowledge in AlphaZerocited by 123
  4. 2022Probing for the Usage of Grammatical Numbercited by 59
  5. 2023Dissociating language and thought in large language modelscited by 29
  6. 2022Naturalistic Causal Probing for Morpho-Syntaxcited by 19
  7. 2023Operationalising Representation in Natural Language Processingcited by 14
  8. 2022Predicting Fine-Tuning Performance with Probingcited by 13
  9. 2021Probing as Quantifying Inductive Biascited by 12
  10. 2023Probing Graph Representationscited by 10
  11. 2026How Vision Becomes Language: A Layer-wise Information-Theoretic Analysis of Multimodal Reasoningcited by 8
  12. 2022SocioProbe: What, When, and Where Language Models Learn about Sociodemographicscited by 7
10 more
  1. 2022Trustworthy Social Bias Measurementcited by 7
  2. 2022The Architectural Bottleneck Principlecited by 5
  3. 2022On the data requirements of probingcited by 4
  4. 2026Operationalising the Superficial Alignment Hypothesis via Task Complexitycited by 4
  5. 2021Quantifying the Task-Specific Information in Text-Based Classificationscited by 3
  6. 2023Measuring Information in Text Explanationscited by 1
  7. 2023A State-Vector Framework for Dataset Effectscited by 1
  8. 2024Using Shapley interactions to understand how models use structurecited by 1
  9. 2024Latent Causal Probing: A Formal Perspective on Probing with Causal Models of Datacited by 1
  10. 2023Rethinking the Construction of Effective Metrics for Understanding the Mechanisms of Pretrained Language Modelscited by 0