# One-shot Entropy Minimization

arXiv: https://arxiv.org/abs/2505.20282 · HTML: https://mlsanity.com/paper/2505.20282/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 23 references that are in the map.
- First posted on arXiv 26 May 2025.

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild](https://mlsanity.com/paper/2503.18892/) (2025)
- [Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model](https://mlsanity.com/paper/2503.24290/) (2025)
- [Reinforcement Learning for Reasoning in Large Language Models with One Training Example](https://mlsanity.com/paper/2504.20571/) (2025)
- [The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning](https://mlsanity.com/paper/2505.15134/) (2025)
- [On Memorization of Large Language Models in Logical Reasoning](https://mlsanity.com/paper/2410.23123/) (2024)
- [Interpretable Contrastive Monte Carlo Tree Search Reasoning](https://mlsanity.com/paper/2410.01707/) (2024)
- [Mixed Preference Optimization: Reinforcement Learning with Data Selection and Better Reference Model](https://mlsanity.com/paper/2403.19443/) (2024)

## Built on by (most cited first, 30 of 32)

- [Spurious Rewards: Rethinking Training Signals in RLVR](https://mlsanity.com/paper/2506.10947/) (2025)
- [Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination](https://mlsanity.com/paper/2507.10532/) (2025)
- [Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR](https://mlsanity.com/paper/2508.14029/) (2025)
- [Prosperity before Collapse: How Far Can Off-Policy RL Reach with Stale Data on LLMs?](https://mlsanity.com/paper/2510.01161/) (2025)
- [Harnessing Uncertainty: Entropy-Modulated Policy Gradients for Long-Horizon LLM Agents](https://mlsanity.com/paper/2509.09265/) (2025)
- [EDGE-GRPO: Entropy-Driven GRPO with Guided Error Correction for Advantage Diversity](https://mlsanity.com/paper/2507.21848/) (2025)
- [No Free Lunch: Rethinking Internal Feedback for LLM Reasoning](https://mlsanity.com/paper/2506.17219/) (2025)
- [Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzles](https://mlsanity.com/paper/2505.23590/) (2025)
- [Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Reward](https://mlsanity.com/paper/2512.16912/) (2025)
- [Low-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Reward](https://mlsanity.com/paper/2510.03222/) (2025)
- [Differential Smoothing Mitigates Sharpening and Improves LLM Reasoning](https://mlsanity.com/paper/2511.19942/) (2025)
- [Revisiting LLM Reasoning via Information Bottleneck](https://mlsanity.com/paper/2507.18391/) (2025)
- [Compute as Teacher: Turning Inference Compute Into Reference-Free Supervision](https://mlsanity.com/paper/2509.14234/) (2025)
- [Incentivizing LLMs to Self-Verify Their Answers](https://mlsanity.com/paper/2506.01369/) (2025)
- [EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RL](https://mlsanity.com/paper/2605.18703/) (2026)
- [Know When to Explore: Difficulty-Aware Certainty as a Guide for LLM Reinforcement Learning](https://mlsanity.com/paper/2509.00125/) (2025)
- [Clip-Low Increases Entropy and Clip-High Decreases Entropy in Reinforcement Learning of Large Language Models](https://mlsanity.com/paper/2509.26114/) (2025)
- [RiskPO: Risk-based Policy Optimization via Verifiable Reward for LLM Post-Training](https://mlsanity.com/paper/2510.00911/) (2025)
- [Likelihood-Based Reward Designs for General LLM Reasoning](https://mlsanity.com/paper/2602.03979/) (2026)
- [From Exploration to Exploitation: A Two-Stage Entropy RLVR Approach for Noise-Tolerant MLLM Training](https://mlsanity.com/paper/2511.07738/) (2025)
- [Explore Briefly, Then Decide: Mitigating LLM Overthinking via Cumulative Entropy Regulation](https://mlsanity.com/paper/2510.02249/) (2025)
- [PowerFlow: Unlocking the Dual Nature of LLMs via Principled Distribution Matching](https://mlsanity.com/paper/2603.18363/) (2026)
- [Spectral Logit Sculpting: Adaptive Low-Rank Logit Transformation for Controlled Text Generation](https://mlsanity.com/paper/2509.25204/) (2025)
- [Experience Sharing in Mutual Reinforcement Learning for Heterogeneous Language Models](https://mlsanity.com/paper/2605.07244/) (2026)
- [THIRDEYE: Cue-Aware Monocular Depth Estimation via Brain-Inspired Multi-Stage Fusion](https://mlsanity.com/paper/2506.20877/) (2025)
- [Revisiting the UID Hypothesis in LLM Reasoning Traces](https://mlsanity.com/paper/2510.13850/) (2025)
- [Test-time RL alignment exposes task familiarity artifacts in LLM benchmarks](https://mlsanity.com/paper/2603.12875/) (2026)
- [SELF-EMO: Emotional Self-Evolution from Recognition to Consistent Expression](https://mlsanity.com/paper/2604.18003/) (2026)
- [TEMPO: Scaling Test-time Training for Large Reasoning Models](https://mlsanity.com/paper/2604.19295/) (2026)
- [Rethinking Entropy Minimization in Test-Time Adaptation for Autoregressive Models](https://mlsanity.com/paper/2605.08186/) (2026)

## Abstract (arXiv)

We trained 13,440 large language models and found that entropy minimization requires only a single unlabeled data and 10 steps optimization to achieve performance improvements comparable to or even greater than those obtained using thousands of data and carefully designed rewards in rule-based reinforcement learning. This striking result may prompt a rethinking of post-training paradigms for large language models. Our code is avaliable at https://github.com/zitian-gao/one-shot-em.