# Putting words in context: LSTM language models and lexical ambiguity

arXiv: https://arxiv.org/abs/1906.05149 · HTML: https://mlsanity.com/paper/1906.05149/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2019.

## Built on by (most cited first, 7 of 7)

- [Word Acquisition in Neural Language Models](https://mlsanity.com/paper/2110.02406/) (2021)
- [Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Senses](https://mlsanity.com/paper/2104.14694/) (2021)
- [Investigating semantic subspaces of Transformer sentence embeddings through linear structural probing](https://mlsanity.com/paper/2310.11923/) (2023)
- [RAW-C: Relatedness of Ambiguous Words--in Context (A New Lexical Resource for English)](https://mlsanity.com/paper/2105.13266/) (2021)
- [Exploring the Representation of Word Meanings in Context: A Case Study on Homonymy and Synonymy](https://mlsanity.com/paper/2106.13553/) (2021)
- [Extending an Event-type Ontology: Adding Verbs and Classes Using Fine-tuned LLMs Suggestions](https://mlsanity.com/paper/2306.02130/) (2023)
- [Measuring Context-Word Biases in Lexical Semantic Datasets](https://mlsanity.com/paper/2112.06733/) (2021)

## Abstract (arXiv)

In neural network models of language, words are commonly represented using context-invariant representations (word embeddings) which are then put in context in the hidden layers. Since words are often ambiguous, representing the contextually relevant information is not trivial. We investigate how an LSTM language model deals with lexical ambiguity in English, designing a method to probe its hidden representations for lexical and contextual information about words. We find that both types of information are represented to a large extent, but also that there is room for improvement for contextual information.