# Correlation Coefficients and Semantic Textual Similarity

arXiv: https://arxiv.org/abs/1905.07790 · HTML: https://mlsanity.com/paper/1905.07790/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2019.

## Built on by (most cited first, 20 of 20)

- [All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality](https://mlsanity.com/paper/2109.04404/) (2021)
- [Just Rank: Rethinking Evaluation with Word and Sentence Similarities](https://mlsanity.com/paper/2203.02679/) (2022)
- [Representation Of Lexical Stylistic Features In Language Models' Embedding Space](https://mlsanity.com/paper/2305.18657/) (2023)
- [Trustworthy Social Bias Measurement](https://mlsanity.com/paper/2212.11672/) (2022)
- [Exploring Dimensionality Reduction Techniques in Multilingual Transformers](https://mlsanity.com/paper/2204.08415/) (2022)
- [BanglaAutoKG: Automatic Bangla Knowledge Graph Construction with Semantic Neural Graph Filtering](https://mlsanity.com/paper/2404.03528/) (2024)
- [Semantic Caching of Contextual Summaries for Efficient Question-Answering with Language Models](https://mlsanity.com/paper/2505.11271/) (2025)
- [Semantics or spelling? Probing contextual word embeddings with orthographic noise](https://mlsanity.com/paper/2408.04162/) (2024)
- [Contextualized Semantic Distance between Highly Overlapped Texts](https://mlsanity.com/paper/2110.01176/) (2021)
- [Vocabulary embeddings organize linguistic structure early in language model training](https://mlsanity.com/paper/2510.07613/) (2025)
- [A New Sentence Ordering Method Using BERT Pretrained Model](https://mlsanity.com/paper/2108.11994/) (2021)
- [Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Models](https://mlsanity.com/paper/2308.08774/) (2023)
- [Co-occurrences using Fasttext embeddings for word similarity tasks in Urdu](https://mlsanity.com/paper/2102.10957/) (2021)
- [Recognition of Mental Adjectives in An Efficient and Automatic Style](https://mlsanity.com/paper/2307.11767/) (2023)
- [Language-agnostic, automated assessment of listeners' speech recall using large language models](https://mlsanity.com/paper/2503.01045/) (2025)
- [A Framework for Lightweight Responsible Prompting Recommendation](https://mlsanity.com/paper/2504.08757/) (2025)
- [Intrinsic vs. Extrinsic Evaluation of Czech Sentence Embeddings: Semantic Relevance Doesn't Help with MT Evaluation](https://mlsanity.com/paper/2506.20203/) (2025)
- [KurdSTS: The Kurdish Semantic Textual Similarity](https://mlsanity.com/paper/2510.02336/) (2025)
- [Task-Specific Distance Correlation Matching for Few-Shot Action Recognition](https://mlsanity.com/paper/2512.11340/) (2025)
- [A Manifold-Aware Topic Modeling Approach via Rank-Based Prototypes](https://mlsanity.com/paper/2609.29630/) (2026)

## Abstract (arXiv)

A large body of research into semantic textual similarity has focused on constructing state-of-the-art embeddings using sophisticated modelling, careful choice of learning signals and many clever tricks. By contrast, little attention has been devoted to similarity measures between these embeddings, with cosine similarity being used unquestionably in the majority of cases. In this work, we illustrate that for all common word vectors, cosine similarity is essentially equivalent to the Pearson correlation coefficient, which provides some justification for its use. We thoroughly characterise cases where Pearson correlation (and thus cosine similarity) is unfit as similarity measure. Importantly, we show that Pearson correlation is appropriate for some word vectors but not others. When it is not appropriate, we illustrate how common non-parametric rank correlation coefficients can be used instead to significantly improve performance. We support our analysis with a series of evaluations on word-level and sentence-level semantic textual similarity benchmarks. On the latter, we show that even the simplest averaged word vectors compared by rank correlation easily rival the strongest deep representations compared by cosine similarity.