# PAC-Private Autoregressive Generation: Calibrating Noise to Ensemble Disagreement

arXiv: https://arxiv.org/abs/2609.05676 · HTML: https://mlsanity.com/paper/2609.05676/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 4 Sep 2026.

## Builds on (most cited first, 22 of 22)

- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [Differentially Private Fine-tuning of Language Models](https://mlsanity.com/paper/2110.06500/) (2021)
- [Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data](https://mlsanity.com/paper/1610.05755/) (2016)
- [Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds](https://mlsanity.com/paper/1605.02065/) (2016)
- [Scalable Private Learning with PATE](https://mlsanity.com/paper/1802.08908/) (2018)
- [A* Sampling](https://mlsanity.com/paper/1411.0030/) (2014)
- [Reconstructing Training Data with Informed Adversaries](https://mlsanity.com/paper/2201.04845/) (2022)
- [Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generation](https://mlsanity.com/paper/2309.11765/) (2023)
- [Privacy-preserving Prediction](https://mlsanity.com/paper/1803.10266/) (2018)
- [The Trade-Offs of Private Prediction](https://mlsanity.com/paper/2007.05089/) (2020)
- [Differentially Private Next-Token Prediction of Large Language Models](https://mlsanity.com/paper/2403.15638/) (2024)
- [PAC Privacy: Automatic Privacy Measurement and Control of Data Processing](https://mlsanity.com/paper/2210.03458/) (2022)
- [Adaptively Private Next-Token Prediction of Large Language Models](https://mlsanity.com/paper/2410.02016/) (2024)
- [Residual-PAC Privacy: Automatic Privacy Control Beyond the Gaussian Barrier](https://mlsanity.com/paper/2506.06530/) (2025)
- [PAC to the Future: Zero-Knowledge Proofs of PAC Private Systems](https://mlsanity.com/paper/2602.11954/) (2026)

## Built on by (most cited first, 1 of 1)

- [Protecting Sensitive Data in Image Synthesis via PAC-Private Adaptation for Diffusion Models](https://mlsanity.com/paper/2610.04038/) (2026)

## Abstract (arXiv)

Language models adapted on private text are often served through APIs, so privacy leakage occurs through generated outputs rather than exposed weights. Private prediction protects these releases. Methods such as PMixED incur privacy cost at each release and increasingly rely on the public model over long horizons. PAC privacy instead calibrates noise to output variability across possible secrets, adding less noise when predictions are stable. To our knowledge, PAC-private prediction has not previously been extended from classification to autoregressive generation. We construct m=128 overlapping worlds from the private corpus, with each record appearing in exactly m/2 worlds, and train one adapter per world over a frozen public model. The realized world is the secret. At each token, the public model defines a candidate set, the worlds vote, and their posterior-weighted disagreement determines the PAC noise; unanimity requires no calibration noise. We prove I(S;Y1:T) ≤ I(S;HT) ≤ bT. Our contributions are extending PAC privacy to autoregressive generation, handling adaptive self-generated contexts, and introducing coupled decoding that preserves privacy accounting while avoiding greedy degeneration. On WikiText-103 with GPT-2-small, we retain 74% of the fine-tuning gain at a per-token budget of 2-32, while membership-inference success is bounded by 51.08% after 106 tokens; posterior-entropy estimates of leakage are roughly 17% of the charged budget. Inference privacy is not content protection: even when membership advantage on a memorized canary is indistinguishable from zero, the canary is emitted at the same rate. Against PMixED under matched membership-inference bounds on the same data universe and test set, we retain 98% of non-private headroom from 102 to 106 tokens, versus at most 56%, with no crossover.