mlsanityData to 14 Oct 2026

Auxiliary-Hyperparameter-Free Sampling: Entropy Equilibrium for Text Generation

arXiv 2512.00789 ↗First posted 30 Nov 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Token sampling strategies critically influence text generation quality in large language models (LLMs). However, existing methods introduce additional hyperparameters, requiring extensive tuning and complicating deployment. We present Entropy Equilibrium Sampling (EES), an auxiliary hyperparameter-free approach inspired by information theory that can dynamically adjust candidate sets by balancing normalized entropy with probability mass. We evaluate EES on both reasoning and generation tasks across a range of model architectures. Our results show that EES consistently performs well across temperature settings, delivering competitive accuracy and coherence while maintaining diversity. By eliminating the need for hyperparameter tuning, EES greatly simplifies deployment while improving performance. Code is available at https://github.com/shuanncai/EES

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 37 references that are in the map. First posted on arXiv 30 Nov 2025.

Builds on

28 of 37 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2025Qwen3 Technical Reportcited by 8,820
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  7. 2016Pointer Sentinel Mixture Modelscited by 3,096
  8. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  9. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  10. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  11. 2018Hierarchical Neural Story Generationcited by 1,193
  12. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
16 more
  1. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  2. 2019Neural Text Generation with Unlikelihood Trainingcited by 503
  3. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  4. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 301
  5. 2022Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfercited by 288
  6. 2017Beam Search Strategies for Neural Machine Translationcited by 232
  7. 2024Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputscited by 150
  8. 2022Truncation Sampling as Language Model Desmoothingcited by 123
  9. 2024A Thorough Examination of Decoding Methods in the Era of LLMscited by 114
  10. 2024A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendationscited by 85
  11. 2021Language Model Evaluation Beyond Perplexitycited by 78
  12. 2024Large Language Model Agent for Hyper-Parameter Optimizationcited by 64
  13. 2025MathSmith: Towards Extremely Hard Mathematical Reasoning by Forging Synthetic Problems with a Reinforced Policycited by 24
  14. 2024Improving Open-Ended Text Generation via Adaptive Decodingcited by 21
  15. 2025LexSemBridge: Fine-Grained Dense Representation Enhancement through Token-Aware Embedding Augmentationcited by 5
  16. 2024Mitigating Catastrophic Forgetting in Multi-domain Chinese Spelling Correction by Multi-stage Knowledge Transfer Frameworkcited by 4

Built on by

Not cited yet.