# Word Rotator's Distance

arXiv: https://arxiv.org/abs/2004.15003 · HTML: https://mlsanity.com/paper/2004.15003/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Apr 2020.

## Built on by (most cited first, 30 of 32)

- [Incorporating Residual and Normalization Layers into Analysis of Masked Language Models](https://mlsanity.com/paper/2109.07152/) (2021)
- [Quantifying Context Mixing in Transformers](https://mlsanity.com/paper/2301.12971/) (2023)
- [CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation](https://mlsanity.com/paper/2305.14635/) (2023)
- [Norm of Word Embedding Encodes Information Gain](https://mlsanity.com/paper/2212.09663/) (2022)
- [Semantics at an Angle: When Cosine Similarity Works Until It Doesn't](https://mlsanity.com/paper/2504.16318/) (2025)
- [Sentence Similarity Based on Contexts](https://mlsanity.com/paper/2105.07623/) (2021)
- [Re-evaluating Word Mover's Distance](https://mlsanity.com/paper/2105.14403/) (2021)
- [Trust It or Not: Confidence-Guided Automatic Radiology Report Generation](https://mlsanity.com/paper/2106.10887/) (2021)
- [Approximating 1-Wasserstein Distance with Trees](https://mlsanity.com/paper/2206.12116/) (2022)
- [Extracting Sentence Embeddings from Pretrained Transformer Models](https://mlsanity.com/paper/2408.08073/) (2024)
- [Unbalanced Optimal Transport for Unbalanced Word Alignment](https://mlsanity.com/paper/2306.04116/) (2023)
- [Revisiting Cosine Similarity via Normalized ICA-transformed Embeddings](https://mlsanity.com/paper/2406.10984/) (2024)
- [Revisiting Additive Compositionality: AND, OR and NOT Operations with Word Embeddings](https://mlsanity.com/paper/2105.08585/) (2021)
- [SynWMD: Syntax-aware Word Mover's Distance for Sentence Similarity Evaluation](https://mlsanity.com/paper/2206.10029/) (2022)
- [Unsupervised Sentence Textual Similarity with Compositional Phrase Semantics](https://mlsanity.com/paper/2210.02284/) (2022)
- [Improving Zero-Shot Multilingual Translation with Universal Representations and Cross-Mappings](https://mlsanity.com/paper/2210.15851/) (2022)
- [Improving word mover's distance by leveraging self-attention matrix](https://mlsanity.com/paper/2211.06229/) (2022)
- [Zipfian Whitening](https://mlsanity.com/paper/2411.00680/) (2024)
- [Quantifying Lexical Semantic Shift via Unbalanced Optimal Transport](https://mlsanity.com/paper/2412.12569/) (2024)
- [Subspace Representations for Soft Set Operations and Sentence Similarities](https://mlsanity.com/paper/2210.13034/) (2022)
- [Language on Demand, Knowledge at Core: Composing LLMs with Encoder-Decoder Translation Models for Extensible Multilinguality](https://mlsanity.com/paper/2603.17512/) (2026)
- [Why Mean Pooling Works: Quantifying Second-Order Collapse in Text Embeddings](https://mlsanity.com/paper/2604.27398/) (2026)
- [Word Tour: One-dimensional Word Embeddings via the Traveling Salesman Problem](https://mlsanity.com/paper/2205.01954/) (2022)
- [Norm of Mean Contextualized Embeddings Determines their Variance](https://mlsanity.com/paper/2409.11253/) (2024)
- [Supporting verification of news articles with automated search for semantically similar articles](https://mlsanity.com/paper/2103.15581/) (2021)
- [WMDecompose: A Framework for Leveraging the Interpretable Properties of Word Mover's Distance in Sociocultural Analysis](https://mlsanity.com/paper/2110.07330/) (2021)
- [Moving Other Way: Exploring Word Mover Distance Extensions](https://mlsanity.com/paper/2202.03119/) (2022)
- [Towards human-compatible autonomous car: A study of non-verbal Turing test in automated driving with affective transition modelling](https://mlsanity.com/paper/2212.02908/) (2022)
- [An Empirical Study of Self-supervised Learning with Wasserstein Distance](https://mlsanity.com/paper/2310.10143/) (2023)
- [WRDScore: New Metric for Evaluation of Natural Language Generation Models](https://mlsanity.com/paper/2405.19220/) (2024)

## Abstract (arXiv)

A key principle in assessing textual similarity is measuring the degree of semantic overlap between two texts by considering the word alignment. Such alignment-based approaches are intuitive and interpretable; however, they are empirically inferior to the simple cosine similarity between general-purpose sentence vectors. To address this issue, we focus on and demonstrate the fact that the norm of word vectors is a good proxy for word importance, and their angle is a good proxy for word similarity. Alignment-based approaches do not distinguish them, whereas sentence-vector approaches automatically use the norm as the word importance. Accordingly, we propose a method that first decouples word vectors into their norm and direction, and then computes alignment-based similarity using earth mover's distance (i.e., optimal transport cost), which we refer to as word rotator's distance. Besides, we find how to grow the norm and direction of word vectors (vector converter), which is a new systematic approach derived from sentence-vector estimation methods. On several textual similarity datasets, the combination of these simple proposed methods outperformed not only alignment-based approaches but also strong baselines. The source code is available at https://github.com/eumesy/wrd