# Word Sense Induction with Neural biLM and Symmetric Patterns

arXiv: https://arxiv.org/abs/1808.08518 · HTML: https://mlsanity.com/paper/1808.08518/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Aug 2018.

## Built on by (most cited first, 18 of 18)

- [Leveraging Large Language Models for Multiple Choice Question Answering](https://mlsanity.com/paper/2210.12353/) (2022)
- [An Information-theoretic Approach to Prompt Engineering Without Ground Truth Labels](https://mlsanity.com/paper/2203.11364/) (2022)
- [Characterizing English Variation across Social Media Communities with BERT](https://mlsanity.com/paper/2102.06820/) (2021)
- [Corpus-based Open-Domain Event Type Induction](https://mlsanity.com/paper/2109.03322/) (2021)
- [Always Keep your Target in Mind: Studying Semantics and Improving Performance of Neural Lexical Substitution](https://mlsanity.com/paper/2206.11815/) (2022)
- [Language Models for Lexical Inference in Context](https://mlsanity.com/paper/2102.05331/) (2021)
- [Large Scale Substitution-based Word Sense Induction](https://mlsanity.com/paper/2110.07681/) (2021)
- [PolyLM: Learning about Polysemy through Language Modeling](https://mlsanity.com/paper/2101.10448/) (2021)
- [Multi-Sense Embeddings for Language Models and Knowledge Distillation](https://mlsanity.com/paper/2504.06036/) (2025)
- [BOS at LSCDiscovery: Lexical Substitution for Interpretable Lexical Semantic Change Detection](https://mlsanity.com/paper/2206.11865/) (2022)
- [Verb Sense Clustering using Contextualized Word Representations for Semantic Frame Induction](https://mlsanity.com/paper/2105.13465/) (2021)
- [Multilingual Substitution-based Word Sense Induction](https://mlsanity.com/paper/2405.11086/) (2024)
- [Language models in word sense disambiguation for Polish](https://mlsanity.com/paper/2111.13982/) (2021)
- [Word Sense Induction with Hierarchical Clustering and Mutual Information Maximization](https://mlsanity.com/paper/2210.05422/) (2022)
- [Semantic Cells: Evolutional Process to Acquire Sense Diversity of Items](https://mlsanity.com/paper/2404.14749/) (2024)
- [Analyzing Polysemy Evolution Using Semantic Cells](https://mlsanity.com/paper/2407.16110/) (2024)
- [Deep-change at AXOLOTL-24: Orchestrating WSD and WSI Models for Semantic Change Modeling](https://mlsanity.com/paper/2408.05184/) (2024)
- [Emergent Lexical Semantics in Neural Language Models: Testing Martin's Law on LLM-Generated Text](https://mlsanity.com/paper/2511.21334/) (2025)

## Abstract (arXiv)

An established method for Word Sense Induction (WSI) uses a language model to predict probable substitutes for target words, and induces senses by clustering these resulting substitute vectors. We replace the ngram-based language model (LM) with a recurrent one. Beyond being more accurate, the use of the recurrent LM allows us to effectively query it in a creative way, using what we call dynamic symmetric patterns. The combination of the RNN-LM and the dynamic symmetric patterns results in strong substitute vectors for WSI, allowing to surpass the current state-of-the-art on the SemEval 2013 WSI shared task by a large margin.