mlsanityData to 14 Oct 2026

One Word Is Not Enough: Simple Prompts Improve Word Embeddings

arXiv 2512.06744 ↗First posted 7 Dec 2025cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Text embedding models are designed for sentence-level applications like retrieval and semantic similarity, and are primarily evaluated on sentence-level benchmarks. Their behavior on isolated words is less understood. We show that simply prepending semantic prompts to words before embedding substantially improves word similarity correlations. Testing 7 text embedding models, including text-embedding-3-large (OpenAI), embed-english-v3.0 (Cohere), voyage-3(Voyage AI), all-mpnet-base-v2, and Qwen3-Embedding-8B, on 3 standard benchmarks (SimLex-999, WordSim-353, MEN-3000), we find that prompts like "meaning: {word}" or "Represent the semantic concept: {word}" improve Spearman correlations by up to +0.29 on SimLex-999. Some models fail completely on bare words (correlation = 0) but recover with prompts (+0.73 improvement). Our best results achieve correlation = 0.692 on SimLex-999 with embed-english-v3.0 (Cohere), correlation = 0.811 on WordSim-353, and correlation = 0.855 on MEN-3000 with text-embedding-3-large (OpenAI). These results outperform classic static embeddings like Word2Vec (correlation = 0.40) and even the best static method LexVec (correlation = 0.48) on SimLex-999, establishing a new state-of-the-art for pure embedding methods. This zero-shot technique requires no training and works with any text embedding model.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 16 of its 32 references that are in the map. First posted on arXiv 7 Dec 2025.

Builds on

16 of 32 references.

References · 16

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  3. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  4. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  5. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  6. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  7. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
  8. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
  9. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
  10. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 333
  11. 2014SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimationcited by 172
  12. 2016SimVerb-3500: A Large-Scale Evaluation Set of Verb Similaritycited by 44
4 more
  1. 2016HyperLex: A Large-Scale Evaluation of Graded Lexical Entailmentcited by 26
  2. 2024Revisiting Word Embeddings in the LLM Eracited by 13
  3. 2024The Impact of Word Splitting on the Semantic Content of Contextualized Word Representationscited by 6
  4. 2016Matrix Factorization using Window Sampling and Negative Sampling for Improved Word Representationscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.