# Understanding and Improving Multi-Sense Word Embeddings via Extended Robust Principal Component Analysis

arXiv: https://arxiv.org/abs/1803.01255 · HTML: https://mlsanity.com/paper/1803.01255/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2018.

## Built on by (most cited first, 1 of 1)

- [Compressing Sentence Representation for Semantic Retrieval via Homomorphic Projective Distillation](https://mlsanity.com/paper/2203.07687/) (2022)

## Abstract (arXiv)

Unsupervised learned representations of polysemous words generate a large of pseudo multi senses since unsupervised methods are overly sensitive to contextual variations. In this paper, we address the pseudo multi-sense detection for word embeddings by dimensionality reduction of sense pairs. We propose a novel principal analysis method, termed Ex-RPCA, designed to detect both pseudo multi senses and real multi senses. With Ex-RPCA, we empirically show that pseudo multi senses are generated systematically in unsupervised method. Moreover, the multi-sense word embeddings can by improved by a simple linear transformation based on Ex-RPCA. Our improved word embedding outperform the original one by 5.6 points on Stanford contextual word similarity (SCWS) dataset. We hope our simple yet effective approach will help the linguistic analysis of multi-sense word embeddings in the future.