mlsanityData to 14 Oct 2026

Advancing Decoding Strategies: Enhancements in Locally Typical Sampling for LLMs

arXiv 2506.05387 ↗First posted 3 Jun 2025cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

This chapter explores advancements in decoding strategies for large language models (LLMs), focusing on enhancing the Locally Typical Sampling (LTS) algorithm. Traditional decoding methods, such as top-k and nucleus sampling, often struggle to balance fluency, diversity, and coherence in text generation. To address these challenges, Adaptive Semantic-Aware Typicality Sampling (ASTS) is proposed as an improved version of LTS, incorporating dynamic entropy thresholding, multi-objective scoring, and reward-penalty adjustments. ASTS ensures contextually coherent and diverse text generation while maintaining computational efficiency. Its performance is evaluated across multiple benchmarks, including story generation and abstractive summarization, using metrics such as perplexity, MAUVE, and diversity scores. Experimental results demonstrate that ASTS outperforms existing sampling techniques by reducing repetition, enhancing semantic alignment, and improving fluency.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 70 of its 90 references that are in the map. First posted on arXiv 3 Jun 2025.

Builds on

70 of 90 references.

References · 70

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  4. 2016Pointer Sentinel Mixture Modelscited by 3,096
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  7. 2018Hierarchical Neural Story Generationcited by 1,193
  8. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
  9. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  10. 2019Neural Text Generation with Unlikelihood Trainingcited by 503
  11. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  12. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decodingcited by 269
38 more
  1. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 202
  2. 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 171
  3. 2023DistillSpec: Improving Speculative Decoding via Knowledge Distillationcited by 165
  4. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 161
  5. 2018Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overviewcited by 160
  6. 2023Controlled Decoding from Language Modelscited by 135
  7. 2024LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-trainingcited by 123
  8. 2024A Thorough Examination of Decoding Methods in the Era of LLMscited by 114
  9. 2023Online Speculative Decodingcited by 113
  10. 2022Locally Typical Samplingcited by 111
  11. 2024Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLMcited by 109
  12. 2023Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generationcited by 105
  13. 2020Trading Off Diversity and Quality in Natural Language Generationcited by 96
  14. 2022On Decoding Strategies for Neural Text Generatorscited by 90
  15. 2024PEARL: Parallel Speculative Decoding with Adaptive Draft Lengthcited by 85
  16. 2023Contrastive Decoding Improves Reasoning in Large Language Modelscited by 74
  17. 2024OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structurecited by 67
  18. 2024Controllable Text Generation for Large Language Models: A Surveycited by 57
  19. 2023Learning to Generate Better Than Your LLMcited by 49
  20. 2024Enhancing Contextual Understanding in Large Language Models through Contrastive Decodingcited by 49
  21. 2023Click: Controllable Text Generation with Sequence Likelihood Contrastive Learningcited by 41
  22. 2024Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Freecited by 37
  23. 2022MAUVE Scores for Generative Models: Theory and Practicecited by 36
  24. 2020Semi-Autoregressive Training Improves Mask-Predict Decodingcited by 33
  25. 2024Dynamic Speculation Lookahead Accelerates Speculative Decoding of Large Language Modelscited by 31
  26. 2020A Systematic Characterization of Sampling Algorithms for Open-ended Language Generationcited by 28
  27. 2023Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decodingcited by 28
  28. 2024Recursive Speculative Decoding: Accelerating LLM Inference via Sampling Without Replacementcited by 27
  29. 2024Nearest Neighbor Speculative Decoding for LLM Generation and Attributioncited by 25
  30. 2023BOLT: Fast Energy-based Controlled Text Generation with Tunable Biasescited by 24
  31. 2024APAR: LLMs Can Do Auto-Parallel Auto-Regressive Decodingcited by 24
  32. 2024FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skippingcited by 23
  33. 2024Direct Alignment of Draft Model for Speculative Decoding with Chat-Fine-Tuned LLMscited by 22
  34. 2022CoNT: Contrastive Neural Text Generationcited by 21
  35. 2022Arithmetic Sampling: Parallel Diverse Decoding for Large Language Modelscited by 19
  36. 2024Toward Inference-optimal Mixture-of-Expert Large Language Modelscited by 19
  37. 2024Entropy-Based Decoding for Retrieval-Augmented Large Language Modelscited by 19
  38. 2024Distillation Contrastive Decoding: Improving LLMs Reasoning with Contrastive Decoding and Distillationcited by 16

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025Geometric Metrics and LLMs: What They Measure and When They Workcited by 4
  2. 2025Multi-Amateur Contrastive Decoding for Text Generationcited by 3
  3. 2025Hierarchical Verification of Speculative Beams for Accelerating LLM Inferencecited by 1
  4. 2025Confidence-Modulated Speculative Decoding for Large Language Modelscited by 1