Semantics or spelling? Probing contextual word embeddings with orthographic noise
Not on the map. Why?
Abstract (arXiv)
Pretrained language model (PLM) hidden states are frequently employed as contextual word embeddings (CWE): high-dimensional representations that encode semantic information given linguistic context. Across many areas of computational linguistics research, similarity between CWEs is interpreted as semantic similarity. However, it remains unclear exactly what information is encoded in PLM hidden states. We investigate this practice by probing PLM representations using minimal orthographic noise. We expect that if CWEs primarily encode semantic information, a single character swap in the input word will not drastically affect the resulting representation,given sufficient linguistic context. Surprisingly, we find that CWEs generated by popular PLMs are highly sensitive to noise in input data, and that this sensitivity is related to subword tokenization: the fewer tokens used to represent a word at input, the more sensitive its corresponding CWE. This suggests that CWEs capture information unrelated to word-level meaning and can be manipulated through trivial modifications of input data. We conclude that these PLM-derived CWEs may not be reliable semantic proxies, and that caution is warranted when interpreting representational similarity
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 21 of its 33 references that are in the map. First posted on arXiv 8 Aug 2024.
Builds on
21 of 33 references.
References · 21
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
- 2016Pointer Sentinel Mixture Modelscited by 2,992
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
- 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
- 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
- 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
- 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
- 2021CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representationcited by 229
9 more
- 2019BPE-Dropout: Simple and Effective Subword Regularizationcited by 189
- 2020Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biasescited by 167
- 2021Charformer: Fast Character Transformers via Gradient-based Subword Tokenizationcited by 158
- 2021All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Qualitycited by 150
- 2021Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLPcited by 130
- 2021BERT Busters: Outlier Dimensions that Disrupt Transformerscited by 110
- 2019Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translationcited by 52
- 2022Problems with Cosine as a Measure of Embedding Similarity for High Frequency Wordscited by 49
- 2019Correlation Coefficients and Semantic Textual Similaritycited by 20
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.