# A visual observation on the geometry of UMAP projections of the difference vectors of antonym and synonym word pair embeddings

arXiv: https://arxiv.org/abs/2603.24150 · HTML: https://mlsanity.com/paper/2603.24150/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 23 references that are in the map.
- First posted on arXiv 25 Mar 2026.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [The Platonic Representation Hypothesis](https://mlsanity.com/paper/2405.07987/) (2024)
- [Counter-fitting Word Vectors to Linguistic Constraints](https://mlsanity.com/paper/1603.00892/) (2016)
- [Harnessing the Universal Geometry of Embeddings](https://mlsanity.com/paper/2505.12540/) (2025)
- [Distinguishing Antonyms and Synonyms in a Pattern-based Neural Network](https://mlsanity.com/paper/1701.02962/) (2017)
- [Integrating Distributional Lexical Contrast into Word Embeddings for Antonym-Synonym Distinction](https://mlsanity.com/paper/1605.07766/) (2016)
- [Antonym-Synonym Classification Based on New Sub-space Embeddings](https://mlsanity.com/paper/1906.05612/) (2019)
- [What company do words keep? Revisiting the distributional semantics of J.R. Firth & Zellig Harris](https://mlsanity.com/paper/2205.07750/) (2022)
- [Intuitive Contrasting Map for Antonym Embeddings](https://mlsanity.com/paper/2004.12835/) (2020)

## Abstract (arXiv)

Antonyms, or opposites, are sometimes defined as word pairs that have all of the same contextually relevant properties but one. Seeing how transformer models seem to encode concepts as directions, this begs the question if one can detect ``antonymity'' in the geometry of the embedding vectors of word pairs, especially based on their difference vectors. Such geometrical studies are then naturally contrasted by comparing antonymic pairs to their opposites; synonyms. This paper started as an exploratory project on the complexity of the systems needed to detect the geometry of the embedding vectors of antonymic word pairs. What we now report is a curious ``swirl'' that appears across embedding models in a somewhat specific projection configuration.