# Understanding Privacy Risks of Embeddings Induced by Large Language Models

arXiv: https://arxiv.org/abs/2404.16587 · HTML: https://mlsanity.com/paper/2404.16587/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 51 references that are in the map.
- First posted on arXiv 25 Apr 2024.

## Builds on (most cited first, 16 of 16)

- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Text and Code Embeddings by Contrastive Pre-Training](https://mlsanity.com/paper/2201.10005/) (2022)
- [Beam Search Strategies for Neural Machine Translation](https://mlsanity.com/paper/1702.01806/) (2017)
- [Information Leakage in Embedding Models](https://mlsanity.com/paper/2004.00053/) (2020)
- [Text Embeddings Reveal (Almost) As Much As Text](https://mlsanity.com/paper/2310.06816/) (2023)
- [Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering](https://mlsanity.com/paper/2101.00774/) (2021)
- [Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence](https://mlsanity.com/paper/2305.03010/) (2023)
- [Adapting Large Language Models to Domains via Reading Comprehension](https://mlsanity.com/paper/2309.09530/) (2023)
- [Using Large Language Models for Zero-Shot Natural Language Generation from Knowledge Graphs](https://mlsanity.com/paper/2307.07312/) (2023)
- [Augmenting LLMs with Knowledge: A survey on hallucination prevention](https://mlsanity.com/paper/2309.16459/) (2023)
- [Evaluating Transformer Language Models on Arithmetic Operations Using Number Decomposition](https://mlsanity.com/paper/2304.10977/) (2023)
- [Shared-Private Bilingual Word Embeddings for Neural Machine Translation](https://mlsanity.com/paper/1906.03100/) (2019)
- [Symbolic Querying of Vector Spaces: Probabilistic Databases Meets Relational Embeddings](https://mlsanity.com/paper/2002.10029/) (2020)

## Built on by (most cited first, 3 of 3)

- [Privacy-Preserving LLM Embedding Transmission for End-Cloud Collaboration](https://mlsanity.com/paper/2503.12896/) (2025)
- [Can we generate portable representations for clinical time series data using LLMs?](https://mlsanity.com/paper/2603.23987/) (2026)
- [Vectors Are Not Neutral: Sensitive-Information Inference from Exported LLM Representations in Summarization](https://mlsanity.com/paper/2605.26433/) (2026)

## Abstract (arXiv)

Large language models (LLMs) show early signs of artificial general intelligence but struggle with hallucinations. One promising solution to mitigate these hallucinations is to store external knowledge as embeddings, aiding LLMs in retrieval-augmented generation. However, such a solution risks compromising privacy, as recent studies experimentally showed that the original text can be partially reconstructed from text embeddings by pre-trained language models. The significant advantage of LLMs over traditional pre-trained models may exacerbate these concerns. To this end, we investigate the effectiveness of reconstructing original knowledge and predicting entity attributes from these embeddings when LLMs are employed. Empirical findings indicate that LLMs significantly improve the accuracy of two evaluated tasks over those from pre-trained models, regardless of whether the texts are in-distribution or out-of-distribution. This underscores a heightened potential for LLMs to jeopardize user privacy, highlighting the negative consequences of their widespread use. We further discuss preliminary strategies to mitigate this risk.