mlsanityData to 7 Oct 2026

Probing with Noise: Unpicking the Warp and Weft of Embeddings

arXiv 2210.12206 ↗First posted 21 Oct 2022cs.CL cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Improving our understanding of how information is encoded in vector space can yield valuable interpretability insights. Alongside vector dimensions, we argue that it is possible for the vector norm to also carry linguistic information. We develop a method to test this: an extension of the probing framework which allows for relative intrinsic interpretations of probing results. It relies on introducing noise that ablates information encoded in embeddings, grounded in random baselines and confidence intervals. We apply the method to well-established probing tasks and find evidence that confirms the existence of separate information containers in English GloVe and BERT embeddings. Our correlation analysis aligns with the experimental findings that different encoders use the norm to encode different kinds of information: GloVe stores syntactic and sentence length information in the vector norm, while BERT uses it to encode contextual incongruity.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 43 of its 67 references that are in the map. First posted on arXiv 21 Oct 2022.

Builds on

43 of 67 references.

References · 43

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2018Deep contextualized word representationscited by 2,285
  5. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  6. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  7. 2014Distributed Representations of Sentences and Documentscited by 677
  8. 2019Designing and Interpreting Probes with Control Taskscited by 608
  9. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
  10. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  11. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 382
  12. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
31 more
  1. 2018Analysis Methods in Neural Language Processing: A Surveycited by 276
  2. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 230
  3. 2016Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Taskscited by 216
  4. 2020Information-Theoretic Probing for Linguistic Structurecited by 204
  5. 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 183
  6. 2017What do Neural Machine Translation Models Learn about Morphology?cited by 165
  7. 2015Visualizing and Understanding Recurrent Networkscited by 158
  8. 2019Do NLP Models Know Numbers? Probing Numeracy in Embeddingscited by 157
  9. 2018Dissecting Contextual Word Embeddings: Architecture and Representationcited by 146
  10. 2020With Little Power Comes Great Responsibilitycited by 138
  11. 2019Open Sesame: Getting Inside BERT's Linguistic Knowledgecited by 119
  12. 2020CausaLM: Causal Model Explanation Through Counterfactual Language Modelscited by 117
  13. 2018Identifying and Controlling Important Neurons in Neural Machine Translationcited by 110
  14. 2018Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Informationcited by 97
  15. 2020Analyzing Individual Neurons in Pre-trained Language Modelscited by 97
  16. 2019The emergence of number and syntax units in LSTM language modelscited by 92
  17. 2019BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performancecited by 89
  18. 2021Probing Across Time: What Does RoBERTa Know and When?cited by 73
  19. 2018Evaluating Layers of Representation in Neural Machine Translation on Part-of-Speech and Semantic Tagging Taskscited by 56
  20. 2020Intrinsic Probing through Dimension Selectioncited by 45
  21. 2021Discourse Probing of Pretrained Language Modelscited by 38
  22. 2019Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Textcited by 31
  23. 2022Metaphors in Pre-Trained Language Models: Probing and Generalization Across Datasets and Languagescited by 30
  24. 2019LINSPECTOR: Multilingual Probing Tasks for Word Representationscited by 22
  25. 2015Measuring Word Significance using Distributed Representations of Wordscited by 21
  26. 2022Discovering Salient Neurons in Deep NLP Modelscited by 21
  27. 2018Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysiscited by 17
  28. 2022Probing for Constituency Structure in Neural Language Modelscited by 13
  29. 2020Amnesic Probing: Behavioral Explanation with Amnesic Counterfactualscited by 12
  30. 2019Not All Claims are Created Equal: Choosing the Right Statistical Approach to Assess Hypothesescited by 8
  31. 2018Multi-task Learning for Universal Sentence Embeddings: A Thorough Evaluation using Transfer and Auxiliary Taskscited by 1

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1.

3 papers.

  1. 2023Idioms, Probing and Dangerous Things: Towards Structural Probing for Idiomaticity in Vector Spacecited by 3
  2. 2024Topic Aware Probing: From Sentence Length Prediction to Idiom Identification how reliant are Neural Language Models on Topic?cited by 2
  3. 2023Probing Taxonomic and Thematic Embeddings for Taxonomic Informationcited by 1