# When Text Embedding Meets Large Language Model: A Comprehensive Survey

arXiv: https://arxiv.org/abs/2412.09165 · HTML: https://mlsanity.com/paper/2412.09165/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2024.

## Built on by (most cited first, 19 of 19)

- [The Geometry of Reasoning: Flowing Logics in Representation Space](https://mlsanity.com/paper/2510.09782/) (2025)
- [Can Large Language Models Match Tutoring System Adaptivity? A Benchmarking Study](https://mlsanity.com/paper/2504.05570/) (2025)
- [Contrastive Prompting Enhances Sentence Embeddings in LLMs through Inference-Time Steering](https://mlsanity.com/paper/2505.12831/) (2025)
- [DS-STAR: Data Science Agent for Solving Diverse Tasks across Heterogeneous Formats and Open-Ended Queries](https://mlsanity.com/paper/2509.21825/) (2025)
- [Benchmarking Educational LLMs with Analytics: A Case Study on Gender Bias in Feedback](https://mlsanity.com/paper/2511.08225/) (2025)
- [The Embedder's Dilemma: LLMs Are Better, but at What Cost?](https://mlsanity.com/paper/2608.12875/) (2026)
- [On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Survey](https://mlsanity.com/paper/2507.20783/) (2025)
- [Fine-grained Video Dubbing Duration Alignment with Segment Supervised Preference Optimization](https://mlsanity.com/paper/2508.08550/) (2025)
- [Learning to Compress: Unlocking the Potential of Large Language Models for Text Representation](https://mlsanity.com/paper/2511.17129/) (2025)
- [Gaussian mixture models as a proxy for interacting language models](https://mlsanity.com/paper/2506.00077/) (2025)
- [Med-gte-hybrid: A contextual embedding transformer model for extracting actionable information from clinical texts](https://mlsanity.com/paper/2502.15996/) (2025)
- [DRGW: Learning Disentangled Representations for Robust Graph Watermarking](https://mlsanity.com/paper/2601.13569/) (2026)
- [Latent Abstraction for Retrieval-Augmented Generation](https://mlsanity.com/paper/2604.17866/) (2026)
- [Training-Free versus Training-Based Intent Classification in LLMs: Accuracy, Robustness, and Failure Modes](https://mlsanity.com/paper/2608.02415/) (2026)
- [LLM-BT-Terms: Back-Translation as a Framework for Terminology Standardization and Dynamic Semantic Embedding](https://mlsanity.com/paper/2506.08174/) (2025)
- [Universal Laboratory Model: prognosis of abnormal clinical outcomes based on routine tests](https://mlsanity.com/paper/2506.15330/) (2025)
- [Semi-Supervised Synthetic Data Generation with Fine-Grained Relevance Control for Short Video Search Relevance Modeling](https://mlsanity.com/paper/2509.16717/) (2025)
- [The Pre-Training Study of Expanded-SPLADE Models on Web Document Titles](https://mlsanity.com/paper/2605.01407/) (2026)
- [ReverseEOL: Improving Training-free Text Embeddings via Text Reversal in Decoder-only LLMs](https://mlsanity.com/paper/2606.05858/) (2026)

## Abstract (arXiv)

Text embedding has become a foundational technology in natural language processing (NLP) during the deep learning era, driving advancements across a wide array of downstream tasks. While many natural language understanding challenges can now be modeled using generative paradigms and leverage the robust generative and comprehension capabilities of large language models (LLMs), numerous practical applications - such as semantic matching, clustering, and information retrieval - continue to rely on text embeddings for their efficiency and effectiveness. Therefore, integrating LLMs with text embeddings has become a major research focus in recent years. In this survey, we categorize the interplay between LLMs and text embeddings into three overarching themes: (1) LLM-augmented text embedding, enhancing traditional embedding methods with LLMs; (2) LLMs as text embedders, adapting their innate capabilities for high-quality embedding; and (3) Text embedding understanding with LLMs, leveraging LLMs to analyze and interpret embeddings. By organizing recent works based on interaction patterns rather than specific downstream applications, we offer a novel and systematic overview of contributions from various research and application domains in the era of LLMs. Furthermore, we highlight the unresolved challenges that persisted in the pre-LLM era with pre-trained language models (PLMs) and explore the emerging obstacles brought forth by LLMs. Building on this analysis, we outline prospective directions for the evolution of text embedding, addressing both theoretical and practical opportunities in the rapidly advancing landscape of NLP.