Perturbation is All You Need for Extrapolating Language Models
Not on the map. Why?
Abstract (arXiv)
This paper develops a statistical theory of extrapolation for large language models, by reinterpreting them through pre-post-additive noise models. In contrast to the standard autoregressive next-token prediction based on an exact prefix, we introduce a perturbation-based procedure that first transforms the prefix into a semantic neighbour and then conditions on this perturbed variant for next-token prediction. This yields a hierarchical model with a pre-post-additive noise structure. Within this framework, we develop a rigorous theory of extrapolability, namely, the capacity of a model class to make reliable predictions for token sequences that lie outside the empirical support of the training corpus, by establishing five properties of the proposed procedure: adaptivity, contractivity, robustness, extrapolability, and double robustness. We evaluate the finite sample performance of the proposed procedure using both synthetic and real world language data. Results show that the proposed method consistently improves out-of-support prediction while maintaining competitive in-support performance, demonstrating that perturbation offers a practical route to language modelling.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 35 of its 53 references that are in the map. First posted on arXiv 5 May 2026.
Builds on
35 of 53 references.
References · 35
- 2017Attention Is All You Needcited by 60,361
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
- 2016Pointer Sentinel Mixture Modelscited by 3,096
- 2018Hierarchical Neural Story Generationcited by 1,193
- 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
- 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 138
- 2024On the Algorithmic Bias of Aligning Large Language Models with RLHF: Preference Collapse and Matching Regularizationcited by 72
- 2020A causal framework for distribution generalizationcited by 68
- 2024A Statistical Framework of Watermarks for Large Language Models: Pivot, Detection Efficiency and Optimal Rulescited by 57
23 more
- 2023Engression: Extrapolation through the Lens of Distributional Regressioncited by 51
- 2020GenAug: Data Augmentation for Finetuning Text Generatorscited by 41
- 2022MAUVE Scores for Generative Models: Theory and Practicecited by 36
- 2024Robust Detection of Watermarks for Large Language Models Under Human Editscited by 31
- 2023Tailoring Language Generation Models under Total Variation Distancecited by 27
- 2025Statistical Impossibility and Possibility of Aligning LLMs with Human Preferences: From Condorcet Paradox to Nash Equilibriumcited by 23
- 2022Ranking Inferences Based on the Top Choice of Multiway Comparisonscited by 22
- 2022First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domainscited by 21
- 2026Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statisticcited by 21
- 2024Dual Active Learning for Reinforcement Learning from Human Feedbackcited by 20
- 2024Debiasing Watermarks for Large Language Models via Maximal Couplingcited by 19
- 2023MixCE: Training Autoregressive Language Models by Mixing Forward and Reverse Cross-Entropiescited by 16
- 2025Robust Reinforcement Learning from Human Feedback for Large Language Models Fine-Tuningcited by 13
- 2024Low-Rank Contextual Reinforcement Learning from Heterogeneous Human Feedbackcited by 10
- 2026Detecting LLM-Generated Text with Performance Guaranteescited by 10
- 2025Uncertainty Quantification for Large Language Model Reward Learning under Heterogeneous Human Feedbackcited by 9
- 2023EMO: Earth Mover Distance Optimization for Auto-Regressive Language Modelingcited by 7
- 2025Contextual Online Uncertainty-Aware Preference Learning for Human Feedbackcited by 7
- 2025Uncertainty Quantification for Ranking with Heterogeneous Preferencescited by 6
- 2025Towards Understanding Extrapolation: a Causal Lenscited by 4
- 2026A Statistical Framework for Alignment with Biased AI Feedbackcited by 4
- 2026Privacy-Preserving Reinforcement Learning from Human Feedback via Decoupled Reward Modelingcited by 3
- 2026Kernelized Advantage Estimation: From Nonparametric Statistics to LLM Reasoningcited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.