One Word Is Not Enough: Simple Prompts Improve Word Embeddings
Not on the map. Why?
Abstract (arXiv)
Text embedding models are designed for sentence-level applications like retrieval and semantic similarity, and are primarily evaluated on sentence-level benchmarks. Their behavior on isolated words is less understood. We show that simply prepending semantic prompts to words before embedding substantially improves word similarity correlations. Testing 7 text embedding models, including text-embedding-3-large (OpenAI), embed-english-v3.0 (Cohere), voyage-3(Voyage AI), all-mpnet-base-v2, and Qwen3-Embedding-8B, on 3 standard benchmarks (SimLex-999, WordSim-353, MEN-3000), we find that prompts like "meaning: {word}" or "Represent the semantic concept: {word}" improve Spearman correlations by up to +0.29 on SimLex-999. Some models fail completely on bare words (correlation = 0) but recover with prompts (+0.73 improvement). Our best results achieve correlation = 0.692 on SimLex-999 with embed-english-v3.0 (Cohere), correlation = 0.811 on WordSim-353, and correlation = 0.855 on MEN-3000 with text-embedding-3-large (OpenAI). These results outperform classic static embeddings like Word2Vec (correlation = 0.40) and even the best static method LexVec (correlation = 0.48) on SimLex-999, establishing a new state-of-the-art for pure embedding methods. This zero-shot technique requires no training and works with any text embedding model.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 16 of its 32 references that are in the map. First posted on arXiv 7 Dec 2025.
Builds on
16 of 32 references.
References · 16
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
- 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
- 2015A large annotated corpus for learning natural language inferencecited by 1,847
- 2016Enriching Word Vectors with Subword Informationcited by 1,583
- 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
- 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
- 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
- 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
- 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 333
- 2014SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimationcited by 172
- 2016SimVerb-3500: A Large-Scale Evaluation Set of Verb Similaritycited by 44
4 more
- 2016HyperLex: A Large-Scale Evaluation of Graded Lexical Entailmentcited by 26
- 2024Revisiting Word Embeddings in the LLM Eracited by 13
- 2024The Impact of Word Splitting on the Semantic Content of Contextualized Word Representationscited by 6
- 2016Matrix Factorization using Window Sampling and Negative Sampling for Improved Word Representationscited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026Disentangling Similarity and Relatedness in Topic Modelscited by 0
- 2026Disentangling Similarity and Relatedness in Topic Models11 Mar 2026