# PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restoration

arXiv: https://arxiv.org/abs/2406.01394 · HTML: https://mlsanity.com/paper/2406.01394/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 77 references that are in the map.
- First posted on arXiv 3 Jun 2024.

## Builds on (most cited first, 30 of 52)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Ignore Previous Prompt: Attack Techniques For Language Models](https://mlsanity.com/paper/2211.09527/) (2022)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [Prompt Injection attack against LLM-integrated Applications](https://mlsanity.com/paper/2306.05499/) (2023)
- [ORPO: Monolithic Preference Optimization without Reference Model](https://mlsanity.com/paper/2403.07691/) (2024)
- [Differentially Private Fine-tuning of Language Models](https://mlsanity.com/paper/2110.06500/) (2021)
- [PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance](https://mlsanity.com/paper/2306.05443/) (2023)
- [Text Embeddings Reveal (Almost) As Much As Text](https://mlsanity.com/paper/2310.06816/) (2023)
- [Inspecting and Editing Knowledge Representations in Language Models](https://mlsanity.com/paper/2304.00740/) (2023)
- [Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbations](https://mlsanity.com/paper/1910.08902/) (2019)
- [Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe](https://mlsanity.com/paper/2210.14348/) (2022)
- [TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space](https://mlsanity.com/paper/2402.17811/) (2024)
- [DDXPlus: A New Dataset For Automatic Medical Diagnosis](https://mlsanity.com/paper/2205.09148/) (2022)
- [Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence](https://mlsanity.com/paper/2305.03010/) (2023)
- [When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methods](https://mlsanity.com/paper/2212.10025/) (2022)
- [MPCFormer: fast, performant and private Transformer inference with MPC](https://mlsanity.com/paper/2211.01452/) (2022)
- [Differentially Private Language Models for Secure Data Sharing](https://mlsanity.com/paper/2210.13918/) (2022)

## Built on by (most cited first, 28 of 28)

- [PAPILLON: Privacy Preservation from Internet-based and Local Language Model Ensembles](https://mlsanity.com/paper/2410.17127/) (2024)
- [PrivaCI-Bench: Evaluating Privacy with Contextual Integrity and Legal Compliance](https://mlsanity.com/paper/2502.17041/) (2025)
- [PRISM: Privacy-Aware Routing for Adaptive Cloud-Edge LLM Inference via Semantic Sketch Collaboration](https://mlsanity.com/paper/2511.22788/) (2025)
- [Privacy-Preserving LLM Embedding Transmission for End-Cloud Collaboration](https://mlsanity.com/paper/2503.12896/) (2025)
- [RewardDS: Privacy-Preserving Fine-Tuning for Large Language Models via Reward Driven Data Synthesis](https://mlsanity.com/paper/2502.18517/) (2025)
- [SoK: Privacy-aware LLM in Healthcare: Threat Model, Privacy Techniques, Challenges and Recommendations](https://mlsanity.com/paper/2601.10004/) (2026)
- [Privasis: Synthesizing the Largest "Public" Private Dataset from Scratch](https://mlsanity.com/paper/2602.03183/) (2026)
- [Meta-Learning for Speeding Up Large Model Inference in Decentralized Environments](https://mlsanity.com/paper/2410.21340/) (2024)
- [From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications](https://mlsanity.com/paper/2505.22311/) (2025)
- [Galaxy: A Cognition-Centered Framework for Proactive, Privacy-Preserving, and Self-Evolving LLM Agents](https://mlsanity.com/paper/2508.03991/) (2025)
- [Differentially Private and Communication Efficient Large Language Model Split Inference via Stochastic Quantization and Soft Prompt](https://mlsanity.com/paper/2602.11513/) (2026)
- [Towards Privacy-Preserving LLM Inference via Covariant Obfuscation (Technical Report)](https://mlsanity.com/paper/2603.01499/) (2026)
- [Local-First Intelligence via Proactive SLM Collaborating with Feedback-Providing LLM](https://mlsanity.com/paper/2604.17827/) (2026)
- [Confidential Prompting: Privacy-preserving LLM Inference on Cloud](https://mlsanity.com/paper/2409.19134/) (2024)
- [Burn-After-Use for Preventing Data Leakage through a Secure Multi-Tenant Architecture in Enterprise LLM](https://mlsanity.com/paper/2601.06627/) (2026)
- [Your Inference Request Will Become a Black Box: Confidential Inference for Cloud-based Large Language Models](https://mlsanity.com/paper/2603.00196/) (2026)
- [Towards Privacy-Preserving Large Language Model: Text-free Inference Through Alignment and Adaptation](https://mlsanity.com/paper/2604.06831/) (2026)
- [PrivScope: Task-scoped Disclosure Control for Hybrid Agentic Systems](https://mlsanity.com/paper/2605.16630/) (2026)
- [An Efficient and Privacy-Preserving Architecture for Cross-Institutional Collaborative RAG](https://mlsanity.com/paper/2605.25716/) (2026)
- [Vision Language Model Helps Private Information De-Identification in Vision Data](https://mlsanity.com/paper/2606.09132/) (2026)
- [Demystifying the Privacy-Utility Trade-off in LLM Interactions](https://mlsanity.com/paper/2609.10992/) (2026)
- [Semantic Encryption: Secure and Effective Interaction with Cloud-based Large Language Models via Semantic Transformation](https://mlsanity.com/paper/2508.01638/) (2025)
- [Meta-Learning for Speeding Up Large Model Inference in Decentralized Environments](https://mlsanity.com/paper/2508.09194/) (2025)
- [BodhiPromptShield: Pre-Inference Prompt Mediation for Surface-Form Privacy Propagation in LLM Agent Pipelines](https://mlsanity.com/paper/2604.05793/) (2026)
- [P2Skill: Privacy Preserving Skill Distillation for Cloud-Local LLM Inference Systems](https://mlsanity.com/paper/2608.14094/) (2026)
- [PrivBench: A Holistic and Modular Benchmarking Platform for Evaluating Text-to-Text Privatization](https://mlsanity.com/paper/2608.29624/) (2026)
- [Learning to Refer: Client-Resolved Generation for Privacy-Aware Language Models](https://mlsanity.com/paper/2609.32706/) (2026)
- [PrivacyGuard: Source-Linked Sanitization and Locally Resolvable PII References for Cloud GUI Agents](https://mlsanity.com/paper/2610.16778/) (2026)

## Abstract (arXiv)

The widespread usage of online Large Language Models (LLMs) inference services has raised significant privacy concerns about the potential exposure of private information in user inputs to malicious eavesdroppers. Existing privacy protection methods for LLMs suffer from either insufficient privacy protection, performance degradation, or large inference time overhead. To address these limitations, we propose PrivacyRestore, a plug-and-play method to protect the privacy of user inputs during LLM inference. The server first trains restoration vectors for each privacy span and then release to clients. Privacy span is defined as a contiguous sequence of tokens within a text that contain private information. The client then aggregate restoration vectors of all privacy spans in the input into a single meta restoration vector which is later sent to the server side along with the input without privacy spans.The private information is restored via activation steering during inference. Furthermore, we prove that PrivacyRestore inherently prevents the linear growth of the privacy budget.We create three datasets, covering medical and legal domains, to evaluate the effectiveness of privacy preserving methods. The experimental results show that PrivacyRestore effectively protects private information and maintain acceptable levels of performance and inference overhead.