# Privacy Regularization: Joint Privacy-Utility Optimization in Language Models

arXiv: https://arxiv.org/abs/2103.07567 · HTML: https://mlsanity.com/paper/2103.07567/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 12 Mar 2021.

## Builds on (most cited first, 16 of 16)

- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Differential Privacy Has Disparate Impact on Model Accuracy](https://mlsanity.com/paper/1905.12101/) (2019)
- [Differentially Private Learning Needs Better Features (or Much More Data)](https://mlsanity.com/paper/2011.11660/) (2020)
- [Deep Learning with Gaussian Differential Privacy](https://mlsanity.com/paper/1911.11607/) (2019)
- [Gmail Smart Compose: Real-Time Assisted Writing](https://mlsanity.com/paper/1906.00080/) (2019)
- [A General Approach to Adding Differential Privacy to Iterative Training Procedures](https://mlsanity.com/paper/1812.06210/) (2018)
- [Towards Robust and Privacy-preserving Text Representations](https://mlsanity.com/paper/1805.06093/) (2018)
- [Privacy in Deep Learning: A Survey](https://mlsanity.com/paper/2004.12254/) (2020)
- [Training Production Language Models without Memorizing User Data](https://mlsanity.com/paper/2009.10031/) (2020)
- [Neither Private Nor Fair: Impact of Data Imbalance on Utility and Fairness in Differential Privacy](https://mlsanity.com/paper/2009.06389/) (2020)
- [Fair Decision Making using Privacy-Protected Data](https://mlsanity.com/paper/1905.12744/) (2019)
- [Enabling Fast Differentially Private SGD via Just-in-Time Compilation and Vectorization](https://mlsanity.com/paper/2010.09063/) (2020)
- [Privacy-preserving Neural Representations of Text](https://mlsanity.com/paper/1808.09408/) (2018)

## Built on by (most cited first, 30 of 30)

- [Large Language Models Can Be Strong Differentially Private Learners](https://mlsanity.com/paper/2110.05679/) (2021)
- [Deduplicating Training Data Mitigates Privacy Risks in Language Models](https://mlsanity.com/paper/2202.06539/) (2022)
- [Membership Inference Attacks against Language Models via Neighbourhood Comparison](https://mlsanity.com/paper/2305.18462/) (2023)
- [Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks](https://mlsanity.com/paper/2203.03929/) (2022)
- [Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration](https://mlsanity.com/paper/2311.06062/) (2023)
- [Do Language Models Plagiarize?](https://mlsanity.com/paper/2203.07618/) (2022)
- [Mix and Match: Learning-free Controllable Text Generation using Energy Language Models](https://mlsanity.com/paper/2203.13299/) (2022)
- [A Customized Text Sanitization Mechanism with Differential Privacy](https://mlsanity.com/paper/2207.01193/) (2022)
- [Training Data Extraction From Pre-trained Language Models: A Survey](https://mlsanity.com/paper/2305.16157/) (2023)
- [Membership Inference on Word Embedding and Beyond](https://mlsanity.com/paper/2106.11384/) (2021)
- [Submix: Practical Private Prediction for Large-Scale Language Models](https://mlsanity.com/paper/2201.00971/) (2022)
- [PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs](https://mlsanity.com/paper/2406.02958/) (2024)
- [Benchmarking Differential Privacy and Federated Learning for BERT Models](https://mlsanity.com/paper/2106.13973/) (2021)
- [Navigating LLM Ethics: Advancements, Challenges, and Future Directions](https://mlsanity.com/paper/2406.18841/) (2024)
- [Watermarking Text Data on Large Language Models for Dataset Copyright](https://mlsanity.com/paper/2305.13257/) (2023)
- [Are Chatbots Ready for Privacy-Sensitive Applications? An Investigation into Input Regurgitation and Prompt-Induced Sanitization](https://mlsanity.com/paper/2305.15008/) (2023)
- [Style Pooling: Automatic Text Style Obfuscation for Improved Classification Fairness](https://mlsanity.com/paper/2109.04624/) (2021)
- [You Don't Know My Favorite Color: Preventing Dialogue Representations from Revealing Speakers' Private Personas](https://mlsanity.com/paper/2205.10228/) (2022)
- [Privacy Leakage in Text Classification: A Data Extraction Approach](https://mlsanity.com/paper/2206.04591/) (2022)
- [Detecting Data Contamination in LLMs via In-Context Learning](https://mlsanity.com/paper/2510.27055/) (2025)
- [Deconstructing Classifiers: Towards A Data Reconstruction Attack Against Text Classification Models](https://mlsanity.com/paper/2306.13789/) (2023)
- [Differentially Private Learning Needs Better Model Initialization and Self-Distillation](https://mlsanity.com/paper/2410.17566/) (2024)
- [Learning Obfuscations Of LLM Embedding Sequences: Stained Glass Transform](https://mlsanity.com/paper/2506.09452/) (2025)
- [Using Property Elicitation to Understand the Impacts of Fairness Regularizers](https://mlsanity.com/paper/2309.11343/) (2023)
- [Fragments to Facts: Partial-Information Fragment Inference from LLMs](https://mlsanity.com/paper/2505.13819/) (2025)
- [UNO: Unlearning via Orthogonalization in Generative models](https://mlsanity.com/paper/2506.04712/) (2025)
- [RL-Finetuned LLMs for Privacy-Preserving Synthetic Rewriting](https://mlsanity.com/paper/2508.19286/) (2025)
- [Chain-of-Authorization: Embedding authorization into large language models](https://mlsanity.com/paper/2603.22869/) (2026)
- [FISGuard: Defending Against Membership Inference via Fixed Input Subspaces](https://mlsanity.com/paper/2608.27836/) (2026)
- [The Privacy-Hallucination Tradeoff in Differentially Private Language Models](https://mlsanity.com/paper/2609.00492/) (2026)

## Abstract (arXiv)

Neural language models are known to have a high capacity for memorization of training samples. This may have serious privacy implications when training models on user content such as email correspondence. Differential privacy (DP), a popular choice to train models with privacy guarantees, comes with significant costs in terms of utility degradation and disparate impact on subgroups of users. In this work, we introduce two privacy-preserving regularization methods for training language models that enable joint optimization of utility and privacy through (1) the use of a discriminator and (2) the inclusion of a triplet-loss term. We compare our methods with DP through extensive evaluation. We show the advantages of our regularizers with favorable utility-privacy trade-off, faster training with the ability to tap into existing optimization approaches, and ensuring uniform treatment of under-represented subgroups.