# Rethinking the Privacy of Text Embeddings: A Reproducibility Study of "Text Embeddings Reveal (Almost) As Much As Text"

arXiv: https://arxiv.org/abs/2507.07700 · HTML: https://mlsanity.com/paper/2507.07700/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 53 references that are in the map.
- First posted on arXiv 10 Jul 2025.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Large Dual Encoders Are Generalizable Retrievers](https://mlsanity.com/paper/2112.07899/) (2021)
- [Understanding Deep Image Representations by Inverting Them](https://mlsanity.com/paper/1412.0035/) (2014)
- [Skip-Thought Vectors](https://mlsanity.com/paper/1506.06726/) (2015)
- [Fine-Tuning LLaMA for Multi-Stage Text Retrieval](https://mlsanity.com/paper/2310.08319/) (2023)
- [Information Leakage in Embedding Models](https://mlsanity.com/paper/2004.00053/) (2020)
- [CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims](https://mlsanity.com/paper/2012.00614/) (2020)
- [Text Embeddings Reveal (Almost) As Much As Text](https://mlsanity.com/paper/2310.06816/) (2023)
- [Inverting Visual Representations with Convolutional Networks](https://mlsanity.com/paper/1506.02753/) (2015)
- [Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?](https://mlsanity.com/paper/2104.07762/) (2021)
- [SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrieval](https://mlsanity.com/paper/2207.02578/) (2022)
- [Empowering News Recommendation with Pre-trained Language Models](https://mlsanity.com/paper/2104.07413/) (2021)
- [BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translation](https://mlsanity.com/paper/2109.04588/) (2021)
- [Canary Extraction in Natural Language Understanding Models](https://mlsanity.com/paper/2203.13920/) (2022)
- [Unsupervised Corpus Poisoning Attacks in Continuous Space for Dense Retrieval](https://mlsanity.com/paper/2504.17884/) (2025)
- [Understanding and Mitigating the Threat of Vec2Text to Dense Retrieval Systems](https://mlsanity.com/paper/2402.12784/) (2024)
- [Reproducing HotFlip for Corpus Poisoning Attacks in Dense Retrieval](https://mlsanity.com/paper/2501.04802/) (2025)
- [Enhancing Lexicon-Based Text Embeddings with Large Language Models](https://mlsanity.com/paper/2501.09749/) (2025)
- [SE#PCFG: Semantically Enhanced PCFG for Password Analysis and Cracking](https://mlsanity.com/paper/2306.06824/) (2023)
- [Does Vec2Text Pose a New Corpus Poisoning Threat?](https://mlsanity.com/paper/2410.06628/) (2024)

## Built on by (most cited first, 6 of 6)

- [Embedding-Aware Feature Discovery: Bridging Latent Representations and Interpretable Features in Event Sequences](https://mlsanity.com/paper/2603.15713/) (2026)
- [ctELM: Decoding and Manipulating Embeddings of Clinical Trials with Embedding Language Models](https://mlsanity.com/paper/2601.18796/) (2026)
- [Generating Hierarchical JSON Representations of Scientific Sentences Using LLMs](https://mlsanity.com/paper/2603.23532/) (2026)
- [Hypencoder Revisited: Reproducibility and Analysis of Non-Linear Scoring for First-Stage Retrieval](https://mlsanity.com/paper/2604.27037/) (2026)
- [SHARD: cell-keyed residual splitting for alignment-resistant private dense retrieval](https://mlsanity.com/paper/2606.27976/) (2026)
- [Denoising-Aware Inversion: Revealing Privacy Risks in Noise-Protected Text Embeddings](https://mlsanity.com/paper/2608.18610/) (2026)

## Abstract (arXiv)

Text embeddings are fundamental to many natural language processing (NLP) tasks, extensively applied in domains such as recommendation systems and information retrieval (IR). Traditionally, transmitting embeddings instead of raw text has been seen as privacy-preserving. However, recent methods such as Vec2Text challenge this assumption by demonstrating that controlled decoding can successfully reconstruct original texts from black-box embeddings. The unexpectedly strong results reported by Vec2Text motivated us to conduct further verification, particularly considering the typically non-intuitive and opaque structure of high-dimensional embedding spaces. In this work, we reproduce the Vec2Text framework and evaluate it from two perspectives: (1) validating the original claims, and (2) extending the study through targeted experiments. First, we successfully replicate the original key results in both in-domain and out-of-domain settings, with only minor discrepancies arising due to missing artifacts, such as model checkpoints and dataset splits. Furthermore, we extend the study by conducting a parameter sensitivity analysis, evaluating the feasibility of reconstructing sensitive inputs (e.g., passwords), and exploring embedding quantization as a lightweight privacy defense. Our results show that Vec2Text is effective under ideal conditions, capable of reconstructing even password-like sequences that lack clear semantics. However, we identify key limitations, including its sensitivity to input sequence length. We also find that Gaussian noise and quantization techniques can mitigate the privacy risks posed by Vec2Text, with quantization offering a simpler and more widely applicable solution. Our findings emphasize the need for caution in using text embeddings and highlight the importance of further research into robust defense mechanisms for NLP systems.