mlsanityData to 14 Oct 2026

One-shot Entropy Minimization

arXiv 2505.20282 ↗First posted 26 May 2025cs.CLcited by 32

Not on the map. Why?

Abstract (arXiv)

We trained 13,440 large language models and found that entropy minimization requires only a single unlabeled data and 10 steps optimization to achieve performance improvements comparable to or even greater than those obtained using thousands of data and carefully designed rewards in rule-based reinforcement learning. This striking result may prompt a rethinking of post-training paradigms for large language models. Our code is avaliable at https://github.com/zitian-gao/one-shot-em.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 12 of its 23 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

12 of 23 references.

References · 12

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2021Program Synthesis with Large Language Modelscited by 3,637
  6. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 511
  7. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 461
  8. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 295
  9. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoningcited by 166
  10. 2024On Memorization of Large Language Models in Logical Reasoningcited by 117
  11. 2024Interpretable Contrastive Monte Carlo Tree Search Reasoningcited by 64
  12. 2024Mixed Preference Optimization: Reinforcement Learning with Data Selection and Better Reference Modelcited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 22; 2026: 10.

32 papers.

Sort:

By area:

  1. 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 209
  2. 2025Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contaminationcited by 82
  3. 2025Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVRcited by 54
  4. 2025Prosperity before Collapse: How Far Can Off-Policy RL Reach with Stale Data on LLMs?cited by 53
  5. 2025Harnessing Uncertainty: Entropy-Modulated Policy Gradients for Long-Horizon LLM Agentscited by 46
  6. 2025EDGE-GRPO: Entropy-Driven GRPO with Guided Error Correction for Advantage Diversitycited by 41
  7. 2025No Free Lunch: Rethinking Internal Feedback for LLM Reasoningcited by 37
  8. 2025Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzlescited by 36
  9. 2025Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Rewardcited by 27
  10. 2025Low-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Rewardcited by 25
  11. 2025Differential Smoothing Mitigates Sharpening and Improves LLM Reasoningcited by 22
  12. 2025Revisiting LLM Reasoning via Information Bottleneckcited by 19
20 more
  1. 2025Compute as Teacher: Turning Inference Compute Into Reference-Free Supervisioncited by 18
  2. 2025Incentivizing LLMs to Self-Verify Their Answerscited by 17
  3. 2026EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RLcited by 14
  4. 2025Know When to Explore: Difficulty-Aware Certainty as a Guide for LLM Reinforcement Learningcited by 12
  5. 2025Clip-Low Increases Entropy and Clip-High Decreases Entropy in Reinforcement Learning of Large Language Modelscited by 12
  6. 2025RiskPO: Risk-based Policy Optimization via Verifiable Reward for LLM Post-Trainingcited by 12
  7. 2026Likelihood-Based Reward Designs for General LLM Reasoningcited by 7
  8. 2025From Exploration to Exploitation: A Two-Stage Entropy RLVR Approach for Noise-Tolerant MLLM Trainingcited by 5
  9. 2025Explore Briefly, Then Decide: Mitigating LLM Overthinking via Cumulative Entropy Regulationcited by 4
  10. 2026PowerFlow: Unlocking the Dual Nature of LLMs via Principled Distribution Matchingcited by 2
  11. 2025Spectral Logit Sculpting: Adaptive Low-Rank Logit Transformation for Controlled Text Generationcited by 1
  12. 2026Experience Sharing in Mutual Reinforcement Learning for Heterogeneous Language Modelscited by 1
  13. 2025THIRDEYE: Cue-Aware Monocular Depth Estimation via Brain-Inspired Multi-Stage Fusioncited by 0
  14. 2025Revisiting the UID Hypothesis in LLM Reasoning Tracescited by 0
  15. 2026Test-time RL alignment exposes task familiarity artifacts in LLM benchmarkscited by 0
  16. 2026SELF-EMO: Emotional Self-Evolution from Recognition to Consistent Expressioncited by 0
  17. 2026TEMPO: Scaling Test-time Training for Large Reasoning Modelscited by 0
  18. 2026Rethinking Entropy Minimization in Test-Time Adaptation for Autoregressive Modelscited by 0
  19. 2026What are Key Factors for Updates in RL for LLM Reasoning?cited by 0
  20. 2026Sharpen Before You Adapt: Data-Free Entry-State Sharpening for Test-Time Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Entropy-Preserving Reinforcement Learning
  2. 2025EntroPIC: Towards Stable Long-Term Training of LLMs via Entropy Stabilization with Proportional-Integral Control
  3. 2026Supervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Models
  4. 2026Entropy-Gated Selective Policy Optimization:Token-Level Gradient Allocation for Hybrid Training of Large Language Models
  5. 2026A Comparative Theoretical Analysis of Entropy Control Methods in Reinforcement Learning
  6. 2026Label-Free Reinforcement Learning via Cross-Model Entropy
  7. 2026Entropy Regularization: A Free Correction to Cross-Entropy for Verified Demonstrations
  8. 2025Constrained Entropic Unlearning: A Primal-Dual Framework for Large Language Models