Semantics at an Angle: When Cosine Similarity Works Until It Doesn't
Not on the map. Why?
Abstract (arXiv)
Cosine similarity is a standard comparison rule for learned representations in information retrieval, natural language processing, computer vision, and multimodal learning. Its popularity is well founded: it removes positive radial scale, is computationally convenient, and often matches objectives that train normalized embeddings. These same properties also delimit what cosine can express. Normalization discards radial variation; anisotropic representations can compress angular contrast; high-dimensional neighborhoods can develop hubs; and a symmetric, uncalibrated score may be mismatched to the relation of interest. This article offers a selective review organized around a simple principle: the usefulness of cosine similarity depends jointly on the learned representation, any normalization or post-processing, the scoring rule, and the downstream decision. We derive the main geometric identities, distinguish failure mechanisms that are often conflated, review representative evidence about embedding norms, and describe objective-matched, geometry-aware, hubness-aware, norm-aware, and learned alternatives. The central conclusion is conditional rather than adversarial: cosine's positive-scale invariance is justified when radial variation is nuisance or fixed by the representation contract, but its angular geometry and downstream decision must still be validated.
Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 21 of its 38 references that are in the map. First posted on arXiv 22 Apr 2025.
Builds on
21 of 38 references.
References · 21
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
- 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
- 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
- 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
9 more
- 2017Word Translation Without Parallel Datacited by 487
- 2017All-but-the-Top: Simple and Effective Postprocessing for Word Representationscited by 212
- 2024Is Cosine-Similarity of Embeddings Really About Similarity?cited by 89
- 2021Cross Modal Retrieval with Querybank Normalisationcited by 72
- 2021WhiteningBERT: An Easy Unsupervised Sentence Embedding Approachcited by 69
- 2022Problems with Cosine as a Measure of Embedding Similarity for High Frequency Wordscited by 49
- 2020Word Rotator's Distancecited by 32
- 2022Norm of Word Embedding Encodes Information Gaincited by 21
- 2025On the Importance of Embedding Norms in Self-Supervised Learningcited by 9
Built on by
Citing papers per year: 2025: 4; 2026: 9.
13 papers.
Sort:
By area:
- 2026Geometric Alignment without Functional Equivalence: A Layer-wise Analysis of the Speech-Text Modality Gapcited by 7
- 2025Network-Level Prompt and Trait Leakage in Local Research Agentscited by 5
- 2026Same Patient, Different Words, Different Diagnosis? Evaluating Semantic Stability in Clinical LLMscited by 2
- 2025Grounding the Ungrounded: A Spectral-Graph Framework for Quantifying Hallucinations in Multimodal LLMscited by 1
- 20253DViT-GAT: A Unified Atlas-Based 3D Vision Transformer and Graph Learning Framework for Major Depressive Disorder Detection Using Structural MRI Datacited by 1
- 2025Adversarial Training Improves Generalization Under Distribution Shifts in Bird Sound Classificationcited by 0
- 2026Listen, Attend, Understand: a Regularization Technique for Stable E2E Speech Translation Training on High Variance labelscited by 0
- 2026Distance-to-Distance Ratio: A Similarity Measure for Sentences Based on Rate of Change in LLM Embeddingscited by 0
- 2026A Dual Cross-Attention Graph Learning Framework For Multimodal MRI-Based Major Depressive Disorder Detectioncited by 0
- 2026Machine Learning-Based Detection of MCP Attackscited by 0
- 2026HistoRAG: Embedding Historical Methodology in Retrieval-Augmented Generation Through Critical Technical Practicecited by 0
- 2026Mapping Similarity Spaces across Embedding Models with Synthetic Query Probingcited by 0
- 2026Cosine Similarity Is Not Evidence: Measuring the Noise Floor of Interpretability Transfer Under Quantization12 Aug 2026
- 2026Mapping Similarity Spaces across Embedding Models with Synthetic Query Probing6 Aug 2026
- 2026HistoRAG: Embedding Historical Methodology in Retrieval-Augmented Generation Through Critical Technical Practice16 Jun 2026
- 2026Same Patient, Different Words, Different Diagnosis? Evaluating Semantic Stability in Clinical LLMs28 May 2026
- 2026Machine Learning-Based Detection of MCP Attacks12 Apr 2026
- 2026A Dual Cross-Attention Graph Learning Framework For Multimodal MRI-Based Major Depressive Disorder Detection11 Apr 2026
- 2026Geometric Alignment without Functional Equivalence: A Layer-wise Analysis of the Speech-Text Modality Gap2 Mar 2026
- 2026Distance-to-Distance Ratio: A Similarity Measure for Sentences Based on Rate of Change in LLM Embeddings25 Jan 2026
- 2026Listen, Attend, Understand: a Regularization Technique for Stable E2E Speech Translation Training on High Variance labels3 Jan 2026
- 20253DViT-GAT: A Unified Atlas-Based 3D Vision Transformer and Graph Learning Framework for Major Depressive Disorder Detection Using Structural MRI Data15 Sep 2025
- 2025Network-Level Prompt and Trait Leakage in Local Research Agents27 Aug 2025
- 2025Grounding the Ungrounded: A Spectral-Graph Framework for Quantifying Hallucinations in Multimodal LLMs26 Aug 2025
- 2026Geometric Alignment without Functional Equivalence: A Layer-wise Analysis of the Speech-Text Modality Gapcited by 7
- 2026Same Patient, Different Words, Different Diagnosis? Evaluating Semantic Stability in Clinical LLMscited by 2
- 20253DViT-GAT: A Unified Atlas-Based 3D Vision Transformer and Graph Learning Framework for Major Depressive Disorder Detection Using Structural MRI Datacited by 1
- 2025Adversarial Training Improves Generalization Under Distribution Shifts in Bird Sound Classificationcited by 0
- 2026Listen, Attend, Understand: a Regularization Technique for Stable E2E Speech Translation Training on High Variance labelscited by 0
- 2026Distance-to-Distance Ratio: A Similarity Measure for Sentences Based on Rate of Change in LLM Embeddingscited by 0
- 2026A Dual Cross-Attention Graph Learning Framework For Multimodal MRI-Based Major Depressive Disorder Detectioncited by 0
- 2026Machine Learning-Based Detection of MCP Attackscited by 0
- 2026Cosine Similarity Is Not Evidence: Measuring the Noise Floor of Interpretability Transfer Under Quantizationcited by 0
- 2026Cosine Similarity Is Not Evidence: Measuring the Noise Floor of Interpretability Transfer Under Quantization12 Aug 2026
- 2026Same Patient, Different Words, Different Diagnosis? Evaluating Semantic Stability in Clinical LLMs28 May 2026
- 2026Machine Learning-Based Detection of MCP Attacks12 Apr 2026
- 2026A Dual Cross-Attention Graph Learning Framework For Multimodal MRI-Based Major Depressive Disorder Detection11 Apr 2026
- 2026Geometric Alignment without Functional Equivalence: A Layer-wise Analysis of the Speech-Text Modality Gap2 Mar 2026
- 2026Distance-to-Distance Ratio: A Similarity Measure for Sentences Based on Rate of Change in LLM Embeddings25 Jan 2026
- 2026Listen, Attend, Understand: a Regularization Technique for Stable E2E Speech Translation Training on High Variance labels3 Jan 2026
- 20253DViT-GAT: A Unified Atlas-Based 3D Vision Transformer and Graph Learning Framework for Major Depressive Disorder Detection Using Structural MRI Data15 Sep 2025
- 2025Adversarial Training Improves Generalization Under Distribution Shifts in Bird Sound Classification18 Jul 2025