# Controlled Generation for Private Synthetic Text

arXiv: https://arxiv.org/abs/2509.25729 · HTML: https://mlsanity.com/paper/2509.25729/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 44 references that are in the map.
- First posted on arXiv 30 Sep 2025.

## Builds on (most cited first, 28 of 28)

- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021)
- [TinyStories: How Small Can Language Models Be and Still Speak Coherent English?](https://mlsanity.com/paper/2305.07759/) (2023)
- [Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning](https://mlsanity.com/paper/2310.06694/) (2023)
- [Large Language Models Can Be Strong Differentially Private Learners](https://mlsanity.com/paper/2110.05679/) (2021)
- [Differentially Private Fine-tuning of Language Models](https://mlsanity.com/paper/2110.06500/) (2021)
- [Analysing Mathematical Reasoning Abilities of Neural Models](https://mlsanity.com/paper/1904.01557/) (2019)
- [Differentially Private Generative Adversarial Network](https://mlsanity.com/paper/1802.06739/) (2018)
- [ZeroGen: Efficient Zero-shot Learning via Dataset Generation](https://mlsanity.com/paper/2202.07922/) (2022)
- [Is GPT-3 a Good Data Annotator?](https://mlsanity.com/paper/2212.10450/) (2022)
- [Large Language Models for Code: Security Hardening and Adversarial Testing](https://mlsanity.com/paper/2302.05319/) (2023)
- [How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence](https://mlsanity.com/paper/2004.12158/) (2020)
- [Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbations](https://mlsanity.com/paper/1910.08902/) (2019)
- [GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators](https://mlsanity.com/paper/2006.08265/) (2020)
- [Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe](https://mlsanity.com/paper/2210.14348/) (2022)
- [Differentially Private Language Models for Secure Data Sharing](https://mlsanity.com/paper/2210.13918/) (2022)
- [Harnessing large-language models to generate private synthetic text](https://mlsanity.com/paper/2306.01684/) (2023)
- [The Limits of Word Level Differential Privacy](https://mlsanity.com/paper/2205.02130/) (2022)
- [Locally Differentially Private Document Generation Using Zero Shot Prompting](https://mlsanity.com/paper/2310.16111/) (2023)
- [Leveraging Hierarchical Representations for Preserving Privacy and Utility in Text](https://mlsanity.com/paper/1910.08917/) (2019)
- [ADePT: Auto-encoder based Differentially Private Text Transformation](https://mlsanity.com/paper/2102.01502/) (2021)
- [Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domains](https://mlsanity.com/paper/2410.08327/) (2024)
- [On the Vulnerability of Text Sanitization](https://mlsanity.com/paper/2410.17052/) (2024)
- [SynthTextEval: Synthetic Text Data Generation and Evaluation for High-Stakes Domains](https://mlsanity.com/paper/2507.07229/) (2025)
- [Mention Annotations Alone Enable Efficient Domain Adaptation for Coreference Resolution](https://mlsanity.com/paper/2210.07602/) (2022)

## Built on by (most cited first, 2 of 2)

- [PrivBench: A Holistic and Modular Benchmarking Platform for Evaluating Text-to-Text Privatization](https://mlsanity.com/paper/2608.29624/) (2026)
- [Subgroup Membership Inference Audits of Differentially Private Synthetic Text](https://mlsanity.com/paper/2609.09848/) (2026)

## Abstract (arXiv)

Text anonymization is essential for responsibly developing and deploying AI in high-stakes domains such as healthcare, social services, and law. In this work, we propose a novel methodology for privacy-preserving synthetic text generation that leverages the principles of de-identification and the Hiding In Plain Sight (HIPS) theory. Our approach introduces entity-aware control codes to guide controllable generation using either in-context learning (ICL) or prefix tuning. The ICL variant ensures privacy levels consistent with the underlying de-identification system, while the prefix tuning variant incorporates a custom masking strategy and loss function to support scalable, high-quality generation. Experiments on legal and clinical datasets demonstrate that our method achieves a strong balance between privacy protection and utility, offering a practical and effective solution for synthetic text generation in sensitive domains.