# GenEOL: Harnessing the Generative Power of LLMs for Training-Free Sentence Embeddings

arXiv: https://arxiv.org/abs/2410.14635 · HTML: https://mlsanity.com/paper/2410.14635/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 34 references that are in the map.
- First posted on arXiv 18 Oct 2024.

## Builds on (most cited first, 15 of 15)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty](https://mlsanity.com/paper/2401.15077/) (2024)
- [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://mlsanity.com/paper/2404.05961/) (2024)
- [SentEval: An Evaluation Toolkit for Universal Sentence Representations](https://mlsanity.com/paper/1803.05449/) (2018)
- [An Empirical Model of Large-Batch Training](https://mlsanity.com/paper/1812.06162/) (2018)
- [Generative Representational Instruction Tuning](https://mlsanity.com/paper/2402.09906/) (2024)
- [Large scale distributed neural network training through online distillation](https://mlsanity.com/paper/1804.03235/) (2018)
- [Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verification](https://mlsanity.com/paper/2410.05318/) (2024)
- [BeLLM: Backward Dependency Enhanced Large Language Model for Sentence Embeddings](https://mlsanity.com/paper/2311.05296/) (2023)
- [Simple Techniques for Enhancing Sentence Embeddings in Generative Language Models](https://mlsanity.com/paper/2404.03921/) (2024)
- [Contrastive Learning of Sentence Embeddings from Scratch](https://mlsanity.com/paper/2305.15077/) (2023)
- [How Well Do Text Embedding Models Understand Syntax?](https://mlsanity.com/paper/2311.07996/) (2023)

## Built on by (most cited first, 15 of 15)

- [LLMs are Also Effective Embedding Models: An In-depth Overview](https://mlsanity.com/paper/2412.12591/) (2024)
- [Following the Autoregressive Nature of LLM Embeddings via Compression and Alignment](https://mlsanity.com/paper/2502.11401/) (2025)
- [Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaning](https://mlsanity.com/paper/2506.08354/) (2025)
- [FreeRet: MLLMs as Training-Free Retrievers](https://mlsanity.com/paper/2509.24621/) (2025)
- [SemCSE: Semantic Contrastive Sentence Embeddings Using LLM-Generated Summaries For Scientific Abstracts](https://mlsanity.com/paper/2507.13105/) (2025)
- [PTEB: Towards Robust Text Embedding Evaluation via Stochastic Paraphrasing at Evaluation Time with LLMs](https://mlsanity.com/paper/2510.06730/) (2025)
- [Large Language Models Meet Extreme Multi-label Classification: Scaling and Multi-modal Framework](https://mlsanity.com/paper/2511.13189/) (2025)
- [TermGPT: Multi-Level Contrastive Fine-Tuning for Terminology Adaptation in Legal and Financial Domain](https://mlsanity.com/paper/2511.09854/) (2025)
- [LLM2Vec-Gen: Generative Embeddings from Large Language Models](https://mlsanity.com/paper/2603.10913/) (2026)
- [GASE: Generatively Augmented Sentence Encoding](https://mlsanity.com/paper/2411.04914/) (2024)
- [mEOL: Training-Free Instruction-Guided Multimodal Embedder for Vector Graphics and Image Retrieval](https://mlsanity.com/paper/2604.17054/) (2026)
- [Your UnEmbedding Matrix is Secretly a Feature Lens for Text Embeddings](https://mlsanity.com/paper/2606.07502/) (2026)
- [DocRetriever: A Plug-and-Play Framework for Multimodal Document Retrieval with Comprehensive Benchmark](https://mlsanity.com/paper/2605.30027/) (2026)
- [Selecting What Matters: Semantic Compression-Guided Selective Pooling for Long-Context Embeddings](https://mlsanity.com/paper/2609.37782/) (2026)
- [Lend Me Your Eyes: Instruction-Aware Text Embeddings via Attention Relay](https://mlsanity.com/paper/2610.05564/) (2026)

## Abstract (arXiv)

Training-free embedding methods directly leverage pretrained large language models (LLMs) to embed text, bypassing the costly and complex procedure of contrastive learning. Previous training-free embedding methods have mainly focused on optimizing embedding prompts and have overlooked the benefits of utilizing the generative abilities of LLMs. We propose a novel method, GenEOL, which uses LLMs to generate diverse transformations of a sentence that preserve its meaning, and aggregates the resulting embeddings of these transformations to enhance the overall sentence embedding. GenEOL significantly outperforms the existing training-free embedding methods by an average of 2.85 points across several LLMs on the sentence semantic text similarity (STS) benchmark. GenEOL also achieves notable gains in clustering, reranking, and pair-classification tasks from the MTEB benchmark. Additionally, GenEOL stabilizes representation quality across LLM layers and remains robust to perturbations of embedding prompts.