# MUSE: Modularizing Unsupervised Sense Embeddings

arXiv: https://arxiv.org/abs/1704.04601 · HTML: https://mlsanity.com/paper/1704.04601/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2017.

## Built on by (most cited first, 6 of 6)

- [Driving Context into Text-to-Text Privatization](https://mlsanity.com/paper/2306.01457/) (2023)
- [Learning Sense-Specific Static Embeddings using Contextualised Word Embeddings as a Proxy](https://mlsanity.com/paper/2110.02204/) (2021)
- [PolyLM: Learning about Polysemy through Language Modeling](https://mlsanity.com/paper/2101.10448/) (2021)
- [Towards Multi-Sense Cross-Lingual Alignment of Contextual Embeddings](https://mlsanity.com/paper/2103.06459/) (2021)
- [Mitigating Semantic Leakage in Cross-lingual Embeddings via Orthogonality Constraint](https://mlsanity.com/paper/2409.15664/) (2024)
- [Word Sense Induction with Knowledge Distillation from BERT](https://mlsanity.com/paper/2304.10642/) (2023)

## Abstract (arXiv)

This paper proposes to address the word sense ambiguity issue in an unsupervised manner, where word sense representations are learned along a word sense selection mechanism given contexts. Prior work focused on designing a single model to deliver both mechanisms, and thus suffered from either coarse-grained representation learning or inefficient sense selection. The proposed modular approach, MUSE, implements flexible modules to optimize distinct mechanisms, achieving the first purely sense-level representation learning system with linear-time sense selection. We leverage reinforcement learning to enable joint training on the proposed modules, and introduce various exploration techniques on sense selection for better robustness. The experiments on benchmark data show that the proposed approach achieves the state-of-the-art performance on synonym selection as well as on contextual word similarities in terms of MaxSimC.