mlsanityData to 14 Oct 2026

Text Embeddings Reveal (Almost) As Much As Text

arXiv 2310.06816 ↗First posted 10 Oct 2023cs.CL cs.LGcited by 192

Not on the map. Why?

Cited by 28 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

How much private information do text embeddings reveal about the original text? We investigate the problem of embedding textitinversion, reconstructing the full text represented in dense text embeddings. We frame the problem as controlled generation: generating text that, when reembedded, is close to a fixed point in latent space. We find that although a na"ive model conditioned on the embedding performs poorly, a multi-step method that iteratively corrects and re-embeds text is able to recover 92% of 32-token text inputs exactly. We train our model to decode text embeddings from two state-of-the-art embedding models, and also show that our model can recover important personal information (full names) from a dataset of clinical notes. Our code is available on Github: hrefhttps://github.com/jxmorris12/vec2textgithub.com/jxmorris12/vec2text.

Key facts. Cited by 192 papers in the map (data through 14 Oct 2026). Builds on 40 of its 59 references that are in the map. First posted on arXiv 10 Oct 2023.

Builds on

40 of 59 references.

References · 40

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  6. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  7. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  8. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  9. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  10. 2019Deep Leakage from Gradientscited by 1,021
  11. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  12. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
28 more
  1. 2014Distributed Representations of Sentences and Documentscited by 678
  2. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  3. 2015Generating Sentences from a Continuous Spacecited by 612
  4. 2021ClipCap: CLIP Prefix for Image Captioningcited by 559
  5. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 520
  6. 2021Large Dual Encoders Are Generalizable Retrieverscited by 456
  7. 2014Understanding Deep Image Representations by Inverting Themcited by 403
  8. 2020iDLG: Improved Deep Leakage from Gradientscited by 382
  9. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 376
  10. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 361
  11. 2015Skip-Thought Vectorscited by 322
  12. 2022Generating Sequences by Learning to Self-Correctcited by 277
  13. 2017Toward Controlled Generation of Textcited by 272
  14. 2020Information Leakage in Embedding Modelscited by 203
  15. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 195
  16. 2015Inverting Visual Representations with Convolutional Networkscited by 170
  17. 2018Disentangled Representation Learning for Non-Parallel Text Style Transfercited by 127
  18. 2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118
  19. 2021Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?cited by 93
  20. 2022SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrievalcited by 90
  21. 2023Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentencecited by 81
  22. 2021High Fidelity Visualization of What Your Self-Supervised Representation Knows Aboutcited by 66
  23. 2020Adversarial Semantic Collisionscited by 42
  24. 2020Vec2Face: Unveil Human Faces from their Blackbox Features in Face Recognitioncited by 36
  25. 2022What Are You Token About? Dense Retrieval as Distributions Over the Vocabularycited by 28
  26. 2022LexMAE: Lexicon-Bottlenecked Pretraining for Large-Scale Retrievalcited by 18
  27. 2021Understanding Invariance via Feedforward Inversion of Discriminatively Trained Classifierscited by 11
  28. 2022Decoding a Neural Retriever's Latent Space for Query Suggestioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 47; 2025: 64; 2026: 74.

192 papers.

Sort:

By area:

  1. 2024Generative Representational Instruction Tuningcited by 233
  2. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 127
  3. 2024xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Tokencited by 125
  4. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 111
  5. 2024Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documentscited by 102
  6. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasurescited by 96
  7. 2024The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalitiescited by 69
  8. 2025Harnessing the Universal Geometry of Embeddingscited by 55
  9. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 51
  10. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 46
  11. 2024Extracting Prompts by Inverting LLM Outputscited by 41
  12. 2023PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Modelscited by 37
38 more
  1. 2025The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?cited by 37
  2. 2025Language Models are Injective and Hence Invertiblecited by 35
  3. 2024CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Followingcited by 33
  4. 2024LLMs + Persona-Plug = Personalized LLMscited by 32
  5. 2024PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restorationcited by 28
  6. 2025Cramming 1568 Tokens into a Single Vector and Back Again: Exploring the Limits of Embedding Space Capacitycited by 28
  7. 2024Text Embedding Inversion Security for Multilingual Language Modelscited by 26
  8. 2024Transferable Embedding Inversion Attack: Uncovering Privacy Risks in Text Embeddings without Model Queriescited by 25
  9. 2024Generating Is Believing: Membership Inference Attacks against Retrieval-Augmented Generationcited by 25
  10. 2023Grounding Foundation Models through Federated Transfer Learning: A General Frameworkcited by 24
  11. 2025Shadow in the Cache: Unveiling and Mitigating Privacy Risks of KV-cache in LLM Inferencecited by 21
  12. 2023Sub-Sentence Encoder: Contrastive Learning of Propositional Semantic Representationscited by 20
  13. 2024Leveraging Passage Embeddings for Efficient Listwise Reranking with Large Language Modelscited by 20
  14. 2024GASLITEing the Retrieval: Exploring Vulnerabilities in Dense Embedding-based Searchcited by 20
  15. 2024Contextual Document Embeddingscited by 19
  16. 2023User Modeling in the Era of Large Language Models: Current Research and Future Directionscited by 18
  17. 2024RemoteRAG: A Privacy-Preserving LLM Cloud RAG Servicecited by 17
  18. 2025Security of Internet of Agents: Attacks and Countermeasurescited by 17
  19. 2025Depth Gives a False Sense of Privacy: LLM Internal States Inversioncited by 17
  20. 2024LLoCO: Learning Long Contexts Offlinecited by 16
  21. 2024Diffusion Guided Language Modelingcited by 15
  22. 2025The RAG Paradox: A Black-Box Attack Exploiting Unintentional Vulnerabilities in Retrieval-Augmented Generation Systemscited by 14
  23. 2024A Comparative Analysis of Word-Level Metric Differential Privacy: Benchmarking The Privacy-Utility Trade-offcited by 13
  24. 2024No Free Lunch Theorem for Privacy-Preserving LLM Inferencecited by 13
  25. 2024Model Inversion Attacks: A Survey of Approaches and Countermeasurescited by 11
  26. 2025Unsupervised Corpus Poisoning Attacks in Continuous Space for Dense Retrievalcited by 11
  27. 2024Improving Demonstration Diversity by Human-Free Fusing for Text-to-SQLcited by 10
  28. 2024Information Leakage from Embedding in Large Language Modelscited by 10
  29. 2024Against All Odds: Overcoming Typology, Script, and Language Confusion in Multilingual Embedding Inversion Attackscited by 10
  30. 2025Tricking Retrievers with Influential Tokens: An Efficient Black-Box Corpus Poisoning Attackcited by 10
  31. 2025Do Activation Verbalization Methods Convey Privileged Information?cited by 10
  32. 2024Understanding and Mitigating the Threat of Vec2Text to Dense Retrieval Systemscited by 9
  33. 2024Intruding with Words: Towards Understanding Graph Injection Attacks at the Text Levelcited by 9
  34. 2025Better Language Model Inversion by Compactly Representing Next-Token Distributionscited by 9
  35. 2024DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Modelcited by 8
  36. 2024Eguard: Defending LLM Embeddings Against Inversion Attacks via Text Mutual Information Optimizationcited by 8
  37. 2025Universal Zero-shot Embedding Inversioncited by 8
  38. 2025Privacy-Preserving Federated Embedding Learning for Localized Retrieval-Augmented Generationcited by 8

50 most cited of 192 All 192 (JSON)