mlsanityData to 7 Oct 2026

Embedding Models Measure in Peculiar Ways

arXiv 2609.20821 ↗First posted 17 Sep 2026cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Embedding spaces define notions of semantic similarity and distance. We study whether those embeddings reflect physical measurements of mass, distance, time and volume, which admit a unique, objective notion of semantic equivalence and distance. We find that physical measurement is only weakly modeled in the embedding space, and that instead quite peculiar measurement patterns can be observed. Further analysis indicates that embedding representations of physical measurements are strongly influenced by superficial string similarity, and recalibration of similarity does not substantially improve the alignment.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 35 references that are in the map. First posted on arXiv 17 Sep 2026.

Builds on

22 of 35 references.

References · 22

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  3. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  4. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  5. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  6. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  7. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  8. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  9. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  10. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
  11. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 527
  12. 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 255
10 more
  1. 2025EmbeddingGemma: Powerful and Lightweight Text Representationscited by 128
  2. 2025On the Theoretical Limitations of Embedding-Based Retrievalcited by 98
  3. 2021Representing Numbers in NLP: a Survey and a Visioncited by 83
  4. 2025Interpretable Text Embeddings and Text Similarity Explanation: A Surveycited by 12
  5. 2025Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaningcited by 10
  6. 2025Granite Embedding R2 Modelscited by 7
  7. 2025Sentence Smith: Controllable Edits for Evaluating Text Embeddingscited by 4
  8. 2026DenseOn with the LateOn: Fully Open Dense and Late-Interaction Models for Multilingual, Long-Context, and Code Searchcited by 3
  9. 2026ALEE: Any-Language Evaluation of Embeddings via English-Centric Minimal Pairscited by 1
  10. 2026Exploring Dowker Homology for Sentence Similaritycited by 1

Built on by

Not cited yet.