# On the Efficacy of Sampling Adapters

arXiv: https://arxiv.org/abs/2307.03749 · HTML: https://mlsanity.com/paper/2307.03749/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 45 references that are in the map.
- First posted on arXiv 7 Jul 2023.

## Builds on (most cited first, 30 of 32)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Contrastive Decoding: Open-ended Text Generation as Optimization](https://mlsanity.com/paper/2210.15097/) (2022)
- [Neural Text Generation with Unlikelihood Training](https://mlsanity.com/paper/1908.04319/) (2019)
- [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021)
- [Assessing Generative Models via Precision and Recall](https://mlsanity.com/paper/1806.00035/) (2018)
- [A note on the evaluation of generative models](https://mlsanity.com/paper/1511.01844/) (2015)
- [All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text](https://mlsanity.com/paper/2107.00061/) (2021)
- [Are GANs Created Equal? A Large-Scale Study](https://mlsanity.com/paper/1711.10337/) (2017)
- [Sparse Sequence-to-Sequence Models](https://mlsanity.com/paper/1905.05702/) (2019)
- [Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translation](https://mlsanity.com/paper/2005.10283/) (2020)
- [Unifying Human and Statistical Evaluation for Natural Language Generation](https://mlsanity.com/paper/1904.02792/) (2019)
- [Truncation Sampling as Language Model Desmoothing](https://mlsanity.com/paper/2210.15191/) (2022)
- [Locally Typical Sampling](https://mlsanity.com/paper/2202.00666/) (2022)
- [Trading Off Diversity and Quality in Natural Language Generation](https://mlsanity.com/paper/2004.10450/) (2020)
- [On Decoding Strategies for Neural Text Generators](https://mlsanity.com/paper/2203.15721/) (2022)
- [Training Trajectories of Language Models Across Scales](https://mlsanity.com/paper/2212.09803/) (2022)
- [How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?](https://mlsanity.com/paper/1511.05101/) (2015)
- [Improved Natural Language Generation via Loss Truncation](https://mlsanity.com/paper/2004.14589/) (2020)
- [Text Generation by Learning from Demonstrations](https://mlsanity.com/paper/2009.07839/) (2020)
- [Generalized Entropy Regularization or: There's Nothing Special about Label Smoothing](https://mlsanity.com/paper/2005.00820/) (2020)
- [A Measure-Theoretic Characterization of Tight Language Models](https://mlsanity.com/paper/2212.10502/) (2022)
- [Calibration, Entropy Rates, and Memory in Language Models](https://mlsanity.com/paper/1906.05664/) (2019)
- [A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation](https://mlsanity.com/paper/2009.07243/) (2020)
- [Sparse Text Generation](https://mlsanity.com/paper/2004.02644/) (2020)

## Built on by (most cited first, 17 of 17)

- [From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models](https://mlsanity.com/paper/2406.16838/) (2024)
- [Improving Open-Ended Text Generation via Adaptive Decoding](https://mlsanity.com/paper/2402.18223/) (2024)
- [Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practices](https://mlsanity.com/paper/2408.09169/) (2024)
- [Control the Temperature: Selective Sampling for Diverse and High-Quality LLM Outputs](https://mlsanity.com/paper/2510.01218/) (2025)
- [Pseudo-RIS: Distinctive Pseudo-supervision Generation for Referring Image Segmentation](https://mlsanity.com/paper/2407.07412/) (2024)
- [EMO: Earth Mover Distance Optimization for Auto-Regressive Language Modeling](https://mlsanity.com/paper/2310.04691/) (2023)
- [Local and Global Decoding in Text Generation](https://mlsanity.com/paper/2410.10810/) (2024)
- [Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs](https://mlsanity.com/paper/2510.05987/) (2025)
- [Recurrent Confidence Chain: Temporal-Aware Uncertainty Quantification in Large Language Models](https://mlsanity.com/paper/2601.13368/) (2026)
- [How Sampling Affects the Detectability of Machine-written texts: A Comprehensive Study](https://mlsanity.com/paper/2510.13681/) (2025)
- [MOSAIC: Multiple Observers Spotting AI Content](https://mlsanity.com/paper/2409.07615/) (2024)
- [TempTest: Local Normalization Distortion and the Detection of Machine-generated Text](https://mlsanity.com/paper/2503.20421/) (2025)
- [Agentic AI: The Era of Semantic Decoding](https://mlsanity.com/paper/2403.14562/) (2024)
- [The production of meaning in the processing of natural language](https://mlsanity.com/paper/2603.20381/) (2026)
- [A Probability--Quality Trade-off in Aligned Language Models and its Relation to Sampling Adaptors](https://mlsanity.com/paper/2406.10203/) (2024)
- [Large Language Models Preserve Semantic Isotopies in Story Continuations](https://mlsanity.com/paper/2510.04400/) (2025)
- [Are You Sure You're Sure? On the Impact of Instruction Tuning on Confidence and Lexical Diversity](https://mlsanity.com/paper/2608.13430/) (2026)

## Abstract (arXiv)

Sampling is a common strategy for generating text from probabilistic models, yet standard ancestral sampling often results in text that is incoherent or ungrammatical. To alleviate this issue, various modifications to a model's sampling distribution, such as nucleus or top-k sampling, have been introduced and are now ubiquitously used in language generation systems. We propose a unified framework for understanding these techniques, which we term sampling adapters. Sampling adapters often lead to qualitatively better text, which raises the question: From a formal perspective, how are they changing the (sub)word-level distributions of language generation models? And why do these local changes lead to higher-quality text? We argue that the shift they enforce can be viewed as a trade-off between precision and recall: while the model loses its ability to produce certain strings, its precision rate on desirable text increases. While this trade-off is not reflected in standard metrics of distribution quality (such as perplexity), we find that several precision-emphasizing measures indeed indicate that sampling adapters can lead to probability distributions more aligned with the true distribution. Further, these measures correlate with higher sequence-level quality scores, specifically, Mauve.