# Will it Unblend?

arXiv: https://arxiv.org/abs/2009.09123 · HTML: https://mlsanity.com/paper/2009.09123/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2020.

## Built on by (most cited first, 11 of 11)

- [Superbizarre Is Not Superb: Derivational Morphology Improves BERT's Interpretation of Complex Words](https://mlsanity.com/paper/2101.00403/) (2021)
- [Greed is All You Need: An Evaluation of Tokenizer Inference Methods](https://mlsanity.com/paper/2403.01289/) (2024)
- [Inducing Character-level Structure in Subword-based Language Models with Type-level Interchange Intervention Training](https://mlsanity.com/paper/2212.09897/) (2022)
- [From chocolate bunny to chocolate crocodile: Do Language Models Understand Noun Compounds?](https://mlsanity.com/paper/2305.10568/) (2023)
- [Systematicity in GPT-3's Interpretation of Novel English Noun Compounds](https://mlsanity.com/paper/2210.09492/) (2022)
- [Semantics of Multiword Expressions in Transformer-Based Models: A Survey](https://mlsanity.com/paper/2401.15393/) (2024)
- [A Psycholinguistic Analysis of BERT's Representations of Compounds](https://mlsanity.com/paper/2302.07232/) (2023)
- [Integrating Approaches to Word Representation](https://mlsanity.com/paper/2109.04876/) (2021)
- [Word Boundary Information Isn't Useful for Encoder Language Models](https://mlsanity.com/paper/2401.07923/) (2024)
- [Can AI mimic the human ability to define neologisms?](https://mlsanity.com/paper/2502.14900/) (2025)
- [From sunblock to softblock: Analyzing the correlates of neology in published writing and on social media](https://mlsanity.com/paper/2602.13123/) (2026)

## Abstract (arXiv)

Natural language processing systems often struggle with out-of-vocabulary (OOV) terms, which do not appear in training data. Blends, such as "innoventor", are one particularly challenging class of OOV, as they are formed by fusing together two or more bases that relate to the intended meaning in unpredictable manners and degrees. In this work, we run experiments on a novel dataset of English OOV blends to quantify the difficulty of interpreting the meanings of blends by large-scale contextual language models such as BERT. We first show that BERT's processing of these blends does not fully access the component meanings, leaving their contextual representations semantically impoverished. We find this is mostly due to the loss of characters resulting from blend formation. Then, we assess how easily different models can recognize the structure and recover the origin of blends, and find that context-aware embedding systems outperform character-level and context-free embeddings, although their results are still far from satisfactory.