# Universal Zero-shot Embedding Inversion

arXiv: https://arxiv.org/abs/2504.00147 · HTML: https://mlsanity.com/paper/2504.00147/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 23 references that are in the map.
- First posted on arXiv 31 Mar 2025.

## Builds on (most cited first, 19 of 19)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [Large Dual Encoders Are Generalizable Retrievers](https://mlsanity.com/paper/2112.07899/) (2021)
- [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://mlsanity.com/paper/2404.05961/) (2024)
- [PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models](https://mlsanity.com/paper/2402.07867/) (2024)
- [Information Leakage in Embedding Models](https://mlsanity.com/paper/2004.00053/) (2020)
- [Text Embeddings Reveal (Almost) As Much As Text](https://mlsanity.com/paper/2310.06816/) (2023)
- [Stealing Part of a Production Language Model](https://mlsanity.com/paper/2403.06634/) (2024)
- [From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models](https://mlsanity.com/paper/2406.16838/) (2024)
- [AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models](https://mlsanity.com/paper/2310.15140/) (2023)
- [Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documents](https://mlsanity.com/paper/2406.05870/) (2024)
- [Fast Adversarial Attacks on Language Models In One GPU Minute](https://mlsanity.com/paper/2402.15570/) (2024)
- [Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence](https://mlsanity.com/paper/2305.03010/) (2023)
- [Phantom: General Backdoor Attacks on Retrieval Augmented Language Generation](https://mlsanity.com/paper/2405.20485/) (2024)
- [Extracting Prompts by Inverting LLM Outputs](https://mlsanity.com/paper/2405.15012/) (2024)
- [Transferable Embedding Inversion Attack: Uncovering Privacy Risks in Text Embeddings without Model Queries](https://mlsanity.com/paper/2406.10280/) (2024)
- [Adversarial Decoding: Generating Readable Documents for Adversarial Objectives](https://mlsanity.com/paper/2410.02163/) (2024)

## Built on by (most cited first, 8 of 8)

- [Harnessing the Universal Geometry of Embeddings](https://mlsanity.com/paper/2505.12540/) (2025)
- [Expert Selections In MoE Models Reveal (Almost) As Much As Text](https://mlsanity.com/paper/2602.04105/) (2026)
- [Hybrid privacy-aware semantic search: SVD-truncated document geometry and CKKS-encrypted query reranking under a restricted threat model](https://mlsanity.com/paper/2606.26373/) (2026)
- [Pointing the Way, Hiding the Destination: Practical Private Dense Retrieval at Scale](https://mlsanity.com/paper/2608.25735/) (2026)
- [ObCLIP: Oblivious CLoud-Device Hybrid Image Generation with Privacy Preservation](https://mlsanity.com/paper/2510.04153/) (2025)
- [SHARD: cell-keyed residual splitting for alignment-resistant private dense retrieval](https://mlsanity.com/paper/2606.27976/) (2026)
- [Denoising-Aware Inversion: Revealing Privacy Risks in Noise-Protected Text Embeddings](https://mlsanity.com/paper/2608.18610/) (2026)
- [Learning to Refer: Client-Resolved Generation for Privacy-Aware Language Models](https://mlsanity.com/paper/2609.32706/) (2026)

## Abstract (arXiv)

Embedding inversion, i.e., reconstructing text given its embedding and black-box access to the embedding encoder, is a fundamental problem in both NLP and security. From the NLP perspective, it helps determine how much semantic information about the input is retained in the embedding. From the security perspective, it measures how much information is leaked by vector databases and embedding-based retrieval systems. State-of-the-art methods for embedding inversion, such as vec2text, have high accuracy but require (a) training a separate model for each embedding, and (b) a large number of queries to the corresponding encoder. We design, implement, and evaluate ZSInvert, a zero-shot inversion method based on the recently proposed adversarial decoding technique. ZSInvert is fast, query-efficient, and can be used for any text embedding without training an embedding-specific inversion model. We measure the effectiveness of ZSInvert on several embeddings and demonstrate that it recovers key semantic information about the corresponding texts.