# Word Usage Similarity Estimation with Sentence Representations and Automatic Substitutes

arXiv: https://arxiv.org/abs/1905.08377 · HTML: https://mlsanity.com/paper/1905.08377/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 May 2019.

## Built on by (most cited first, 5 of 5)

- [Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Senses](https://mlsanity.com/paper/2104.14694/) (2021)
- [LLMs as a synthesis between symbolic and distributed approaches to language](https://mlsanity.com/paper/2502.11856/) (2025)
- [The Impact of Word Splitting on the Semantic Content of Contextualized Word Representations](https://mlsanity.com/paper/2402.14616/) (2024)
- [Using Distributional Principles for the Semantic Study of Contextual Language Models](https://mlsanity.com/paper/2111.12174/) (2021)
- [Measuring Context-Word Biases in Lexical Semantic Datasets](https://mlsanity.com/paper/2112.06733/) (2021)

## Abstract (arXiv)

Usage similarity estimation addresses the semantic proximity of word instances in different contexts. We apply contextualized (ELMo and BERT) word and sentence embeddings to this task, and propose supervised models that leverage these representations for prediction. Our models are further assisted by lexical substitute annotations automatically assigned to word instances by context2vec, a neural model that relies on a bidirectional LSTM. We perform an extensive comparison of existing word and sentence representations on benchmark datasets addressing both graded and binary similarity. The best performing models outperform previous methods in both settings.