mlsanityData to 7 Oct 2026

Adaptive Decoding via Latent Preference Optimization

arXiv 2411.09661 ↗First posted 14 Nov 2024cs.CLcited by 9

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

During language model decoding, it is known that using higher temperature sampling gives more creative responses, while lower temperatures are more factually accurate. However, such models are commonly applied to general instruction following, which involves both creative and fact seeking tasks, using a single fixed temperature across all examples and tokens. In this work, we introduce Adaptive Decoding, a layer added to the model to select the sampling temperature dynamically at inference time, at either the token or example level, in order to optimize performance. To learn its parameters we introduce Latent Preference Optimization (LPO) a general approach to train discrete latent variables such as choices of temperature. Our method outperforms all fixed decoding temperatures across a range of tasks that require different temperatures, including UltraFeedback, Creative Story Writing, and GSM8K.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 23 of its 33 references that are in the map. First posted on arXiv 14 Nov 2024.

Builds on

23 of 33 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Calibrating Language Models with Adaptive Temperature Scaling29 Sep
  2. 2024SimPO: Simple Preference Optimization with a Reference-Free Reward23 May
  3. 2023Some things are more CRINGE than others: Iterative Preference Optimization with the Pairwise Cringe Loss27 Dec

Other references · 20

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  6. 2016Pointer Sentinel Mixture Modelscited by 2,992
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  8. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  9. 2018Hierarchical Neural Story Generationcited by 1,182
  10. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  11. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 394
  12. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
8 more
  1. 2024Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Expertscited by 369
  2. 2017Beam Search Strategies for Neural Machine Translationcited by 232
  3. 2024A Thorough Examination of Decoding Methods in the Era of LLMscited by 112
  4. 2020Trading Off Diversity and Quality in Natural Language Generationcited by 94
  5. 2019Calibration of Encoder Decoder Models for Neural Machine Translationcited by 67
  6. 2024EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Samplingcited by 42
  7. 2024Improving Open-Ended Text Generation via Adaptive Decodingcited by 21
  8. 2024Dynamic Stochastic Decoding Strategy for Open-Domain Dialogue Generationcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 2.

9 papers.

  1. 2025Diverse Preference Optimizationcited by 48
  2. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 35
  3. 2025Optimizing Temperature for Language Models with Multi-Sample Inferencecited by 33
  4. 2025Revisiting Self-Consistency from Dynamic Distributional Alignment Perspective on Answer Aggregationcited by 11
  5. 2025Exploring Multi-Temperature Strategies for Token- and Rollout-Level Control in RLVRcited by 8
  6. 2026Learning Adaptive LLM Decodingcited by 2
  7. 2025MultiScale Contextual Bandits for Long Term Objectivescited by 0
  8. 2025On Next-Token Prediction in LLMs: How End Goals Determine the Consistency of Decoding Algorithmscited by 0
  9. 2026Beyond Tokens: A Survey on Decoding Methods for Large Language and Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025p-less Sampling: A Robust Hyperparameter-Free Approach for LLM Decoding
  2. 2024Temperature-Centric Investigation of Speculative Decoding with Knowledge Distillation
  3. 2024Entropy Guided Extrapolative Decoding to Improve Factuality in Large Language Models
  4. 2026Sampling for Quality: Training-Free Reward-Guided LLM Decoding via Sequential Monte Carlo
  5. 2026Adaptive Decoding via Test-Time Policy Learning for Self-Improving Generation
  6. 2026Entropy-informed Decoding: Adaptive Information-Driven Branching
  7. 2026Entropic-Time Inference: Self-Organizing Large Language Model Decoding Beyond Attention
  8. 2025Control the Temperature: Selective Sampling for Diverse and High-Quality LLM Outputs