# Reconstruct Your Previous Conversations! Comprehensively Investigating Privacy Leakage Risks in Conversations with GPT Models

arXiv: https://arxiv.org/abs/2402.02987 · HTML: https://mlsanity.com/paper/2402.02987/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 45 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Builds on (most cited first, 27 of 27)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Ignore Previous Prompt: Attack Techniques For Language Models](https://mlsanity.com/paper/2211.09527/) (2022)
- [Quantifying Memorization Across Neural Language Models](https://mlsanity.com/paper/2202.07646/) (2022)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- ["Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models](https://mlsanity.com/paper/2308.03825/) (2023)
- [Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations](https://mlsanity.com/paper/2310.06387/) (2023)
- [Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory](https://mlsanity.com/paper/2310.17884/) (2023)
- [Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models](https://mlsanity.com/paper/2205.10770/) (2022)
- [MedDialog: Two Large-scale Medical Dialogue Datasets](https://mlsanity.com/paper/2004.03329/) (2020)
- [How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN](https://mlsanity.com/paper/2111.09509/) (2021)
- [Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbations](https://mlsanity.com/paper/1910.08902/) (2019)
- [Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy](https://mlsanity.com/paper/2210.17546/) (2022)
- [The Limits of Word Level Differential Privacy](https://mlsanity.com/paper/2205.02130/) (2022)
- [Locally Differentially Private Document Generation Using Zero Shot Prompting](https://mlsanity.com/paper/2310.16111/) (2023)
- [Data Curation Alone Can Stabilize In-context Learning](https://mlsanity.com/paper/2212.10378/) (2022)
- [Synthetic Data Generation for Grammatical Error Correction with Tagged Corruption Models](https://mlsanity.com/paper/2105.13318/) (2021)
- [TEM: High Utility Metric Differential Privacy on Text](https://mlsanity.com/paper/2107.07928/) (2021)
- [How BPE Affects Memorization in Transformers](https://mlsanity.com/paper/2110.02782/) (2021)

## Built on by (most cited first, 16 of 16)

- [JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMs](https://mlsanity.com/paper/2402.05668/) (2024)
- [MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Models](https://mlsanity.com/paper/2406.07057/) (2024)
- [JADES: A Universal Framework for Jailbreak Assessment via Decompositional Scoring](https://mlsanity.com/paper/2508.20848/) (2025)
- [The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Prompts](https://mlsanity.com/paper/2406.14318/) (2024)
- [SOFT: Selective Data Obfuscation for Protecting LLM Fine-tuning against Membership Inference Attacks](https://mlsanity.com/paper/2506.10424/) (2025)
- [LLM Access Shield: Domain-Specific LLM Framework for Privacy Policy Compliance](https://mlsanity.com/paper/2505.17145/) (2025)
- [Chimera: Harnessing Multi-Agent LLMs for Automatic Insider Threat Simulation](https://mlsanity.com/paper/2508.07745/) (2025)
- [Benchmark of Benchmarks: Unpacking Influence and Code Repository Quality in LLM Safety Benchmarks](https://mlsanity.com/paper/2603.04459/) (2026)
- [Securing RAG: A Risk Assessment and Mitigation Framework](https://mlsanity.com/paper/2505.08728/) (2025)
- [Understanding LLM Behavior When Encountering User-Supplied Harmful Content in Harmless Tasks](https://mlsanity.com/paper/2603.11914/) (2026)
- [Empirical Calibration and Metric Differential Privacy in Language Models](https://mlsanity.com/paper/2503.13872/) (2025)
- [Privacy Guard & Token Parsimony by Prompt and Context Handling and LLM Routing](https://mlsanity.com/paper/2603.28972/) (2026)
- [On the Evidentiary Limits of Membership Inference for Copyright Auditing](https://mlsanity.com/paper/2601.12937/) (2026)
- [The Privacy-Hallucination Tradeoff in Differentially Private Language Models](https://mlsanity.com/paper/2609.00492/) (2026)
- [Detokenization Leaks: Reconstructing Local LLM Outputs From Cache Traces](https://mlsanity.com/paper/2609.06674/) (2026)
- [An Empirical Measurement of Jailbreaking Evaluators](https://mlsanity.com/paper/2609.10594/) (2026)

## Abstract (arXiv)

Significant advancements have recently been made in large language models represented by GPT models. Users frequently have multi-round private conversations with cloud-hosted GPT models for task optimization. Yet, this operational paradigm introduces additional attack surfaces, particularly in custom GPTs and hijacked chat sessions. In this paper, we introduce a straightforward yet potent Conversation Reconstruction Attack. This attack targets the contents of previous conversations between GPT models and benign users, i.e., the benign users' input contents during their interaction with GPT models. The adversary could induce GPT models to leak such contents by querying them with designed malicious prompts. Our comprehensive examination of privacy risks during the interactions with GPT models under this attack reveals GPT-4's considerable resilience. We present two advanced attacks targeting improved reconstruction of past conversations, demonstrating significant privacy leakage across all models under these advanced techniques. Evaluating various defense mechanisms, we find them ineffective against these attacks. Our findings highlight the ease with which privacy can be compromised in interactions with GPT models, urging the community to safeguard against potential abuses of these models' capabilities.