mlsanityData to 7 Oct 2026

Submix: Practical Private Prediction for Large-Scale Language Models

arXiv 2201.00971 ↗First posted 4 Jan 2022cs.LG cs.AI cs.CLcited by 31

Not on the map. Why?

Abstract (arXiv)

Recent data-extraction attacks have exposed that language models can memorize some training samples verbatim. This is a vulnerability that can compromise the privacy of the model's training data. In this work, we introduce SubMix: a practical protocol for private next-token prediction designed to prevent privacy violations by language models that were fine-tuned on a private corpus after pre-training on a public corpus. We show that SubMix limits the leakage of information that is unique to any individual user in the private corpus via a relaxation of group differentially private prediction. Importantly, SubMix admits a tight, data-dependent privacy accounting mechanism, which allows it to thwart existing data-extraction attacks while maintaining the utility of the language model. SubMix is the first protocol that maintains privacy even when publicly releasing tens of thousands of next-token predictions made by large transformer-based models such as GPT-2.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 31 of its 41 references that are in the map. First posted on arXiv 4 Jan 2022.

Builds on

31 of 41 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2016Pointer Sentinel Mixture Modelscited by 2,992
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2020Extracting Training Data from Large Language Modelscited by 2,144
  7. 2018Hierarchical Neural Story Generationcited by 1,182
  8. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  9. 2017Renyi Differential Privacycited by 643
  10. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  11. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
  12. 2018Scalable Private Learning with PATEcited by 266
19 more
  1. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 233
  2. 2015Semi-supervised Sequence Learningcited by 198
  3. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 193
  4. 2019BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarizationcited by 155
  5. 2018Towards Robust and Privacy-preserving Text Representationscited by 75
  6. 2020Training Production Language Models without Memorizing User Datacited by 68
  7. 2020Differentially Private Representation for NLP: Formal Guarantee and An Empirical Study on Privacy and Fairnesscited by 66
  8. 2021Selective Differential Privacy for Language Modelingcited by 61
  9. 2020Individual Privacy Accounting via a Renyi Filtercited by 57
  10. 2018Privacy-preserving Predictioncited by 48
  11. 2020Differentially Private Language Models Benefit from Public Pre-trainingcited by 46
  12. 2018Importance of Search and Evaluation Strategies in Neural Dialogue Modelingcited by 44
  13. 2020A Differentially Private Text Perturbation Method Using a Regularized Mahalanobis Metriccited by 41
  14. 2021Privacy Regularization: Joint Privacy-Utility Optimization in Language Modelscited by 29
  15. 2020The Trade-Offs of Private Predictioncited by 22
  16. 2017R'enyi Differential Privacy Mechanisms for Posterior Samplingcited by 15
  17. 2020Differentially Private Set Unioncited by 13
  18. 2015Dependency Recurrent Neural Language Models for Sentence Completioncited by 9
  19. 2021Differentially Private n-gram Extractioncited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 8; 2024: 9; 2025: 7; 2026: 4.

31 papers.

Sort:

By area:

  1. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  2. 2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 94
  3. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  4. 2023Why Is Public Pretraining Necessary for Private Model Training?cited by 36
  5. 2024CopyBench: Measuring Literal and Non-Literal Reproduction of Copyright-Protected Text in Language Model Generationcited by 35
  6. 2022Differentially Private Decoding in Large Language Modelscited by 34
  7. 2023Identifying and Mitigating Privacy Risks Stemming from Language Models: A Surveycited by 30
  8. 2024Private prediction for large-scale synthetic text generationcited by 24
  9. 2024Differentially Private Next-Token Prediction of Large Language Modelscited by 21
  10. 2023Differentially Private Natural Language Models: Recent Advances and Future Directionscited by 20
  11. 2024Differentially Private Knowledge Distillation via Synthetic Text Generationcited by 18
  12. 2022Differentially Private Model Compressioncited by 13
19 more
  1. 2022Privacy-Preserving Domain Adaptation of Semantic Parserscited by 12
  2. 2023Information Flow Control in Machine Learning through Modular Model Architecturecited by 11
  3. 2024SoK: Reducing the Vulnerability of Fine-tuned Language Models to Membership Inference Attackscited by 10
  4. 2024Data-adaptive Differentially Private Prompt Synthesis for In-Context Learningcited by 10
  5. 2025InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacycited by 10
  6. 2023On the Benefits of Public Representations for Private Transfer Learning under Distribution Shiftcited by 5
  7. 2025Clustering and Median Aggregation Improve Differentially Private Inferencecited by 5
  8. 2025DP-Fusion: Token-Level Differentially Private Inference for Large Language Modelscited by 5
  9. 2024Adaptively Private Next-Token Prediction of Large Language Modelscited by 4
  10. 2024DOMBA: Double Model Balancing for Access-Controlled Language Models via Minimum-Bounded Aggregationcited by 2
  11. 2025Privacy Preserving In-Context-Learning Framework for Large Language Modelscited by 2
  12. 2026EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectorscited by 2
  13. 2025Public Data Assisted Differentially Private In-Context Learningcited by 1
  14. 2025Cocoon: A System Architecture for Differentially Private Training with Correlated Noisescited by 1
  15. 2026Private Prediction via PAC Privacycited by 1
  16. 2023Domain Private Transformers for Multi-Domain Dialog Systemscited by 0
  17. 2025Lower Bounds for Public-Private Learning under Distribution Shiftcited by 0
  18. 2026Chain-of-Authorization: Embedding authorization into large language modelscited by 0
  19. 2026PrivBench: A Holistic and Modular Benchmarking Platform for Evaluating Text-to-Text Privatizationcited by 0