mlsanityData to 14 Oct 2026

ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning

arXiv 2504.21370 ↗First posted 30 Apr 2025cs.AIcited by 72

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Recent models such as OpenAI o1 and DeepSeek-R1 have demonstrated strong performance on reasoning-intensive tasks by generating extended Chain-of-Thought (CoT) traces. While longer reasoning helps with thorough exploration of solution paths for complex problems, it also often leads to inefficient and redundant outputs--a phenomenon commonly described as overthinking. In this paper, we propose ShorterBetter, a simple yet effective reinforcement learning method that enables reasoning models to learn their own optimal CoT lengths without manual supervision. We define the Sample Optimal Length (SOL) as the length of the shortest correct response among multiple generations, which serves as a dynamic reward signal to guide the model toward efficient reasoning. Applied to DeepSeek-Distill-Qwen-1.5B/7B as base models, ShorterBetter achieves 50%-80% reduction in output lengths in both in-domain and out-of-domain reasoning tasks while maintaining accuracy. Our reasoning trace analysis shows that ShorterBetter refines the structure of the reasoning traces by reducing unnecessary repetition, excessive self-verification, and over-exploration of alternatives.

Key facts. Cited by 72 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 48 of its 62 references that are in the map. First posted on arXiv 30 Apr 2025.

Builds on

48 of 62 references.

Closest ancestors · 18

Same direction, newest first

  1. 2025Concise Reasoning via Reinforcement Learning7 Apr
  2. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond27 Mar
  3. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning6 Mar
  4. 2025DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models6 Mar
  5. 2025How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach3 Mar
  6. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning25 Feb
  7. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMs17 Feb
  8. 2025The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Tasks12 Feb
  9. 2025s1: Simple test-time scaling31 Jan
  10. 2025O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning22 Jan
  11. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec
  12. 2024Token-Budget-Aware LLM Reasoning24 Dec
6 more
  1. 2024Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systems12 Dec
  2. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions21 Nov
  3. 2024Training Language Models to Self-Correct via Reinforcement Learning19 Sep
  4. 2024Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost29 Jul
  5. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  6. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar

Other references · 30

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  9. 2024Qwen2.5 Technical Reportcited by 4,942
  10. 2021Program Synthesis with Large Language Modelscited by 3,637
  11. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  12. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
18 more
  1. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  2. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  3. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  4. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  5. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  6. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
  7. 2024A Survey on Large Language Models for Code Generationcited by 658
  8. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  9. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  10. 2024Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Modelscited by 284
  11. 2025Training Language Models to Reason Efficientlycited by 247
  12. 2024Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judgecited by 204
  13. 2025When More is Less: Understanding Chain-of-Thought Length in LLMscited by 183
  14. 2025Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketchingcited by 117
  15. 2025OverThink: Slowdown Attacks on Reasoning LLMscited by 96
  16. 2024An Examination on the Effectiveness of Divide-and-Conquer Prompting in Large Language Modelscited by 12
  17. 2025A Self-Supervised Reinforcement Learning Approach for Fine-Tuning Large Language Models Using Cross-Attention Signalscited by 7
  18. 2024DCR: Divide-and-Conquer Reasoning for Multi-choice Question Answering with LLMscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 32; 2026: 40.

72 papers.

Sort:

By area:

  1. 2026GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimizationcited by 193
  2. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  3. 2025Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RLcited by 62
  4. 2025AdaThinkDrive: Adaptive Thinking via Reinforcement Learning for Autonomous Drivingcited by 58
  5. 2025Optimizing Length Compression in Large Reasoning Modelscited by 38
  6. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 37
  7. 2025OptimalThinkingBench: Evaluating Over and Underthinking in LLMscited by 35
  8. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecyclecited by 35
  9. 2025Learning Unmasking Policies for Diffusion Language Modelscited by 30
  10. 2025Promoting Efficient Reasoning with Verifiable Stepwise Rewardcited by 28
  11. 2025Thinking Fast and Right: Balancing Accuracy and Reasoning Length with Adaptive Rewardscited by 20
  12. 2025Trace Length is a Simple Uncertainty Signal in Reasoning Modelscited by 19
38 more
  1. 2026Does Your Reasoning Model Implicitly Know When to Stop Thinking?cited by 19
  2. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  3. 2025OThink-R1: Intrinsic Fast/Slow Thinking Mode Switching for Over-Reasoning Mitigationcited by 17
  4. 2025DRPO: Efficient Reasoning via Decoupled Reward Policy Optimizationcited by 16
  5. 2025Large Reasoning Embedding Models: Towards Next-Generation Dense Retrieval Paradigmcited by 9
  6. 2026ATTNPO: Attention-Guided Process Supervision for Efficient Reasoningcited by 9
  7. 2026Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learningcited by 9
  8. 2025BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Modelscited by 8
  9. 2026Distilled Reinforcement Learning for LLM Post-trainingcited by 8
  10. 2025R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoningcited by 7
  11. 2025Improving Chain-of-Thought Efficiency for Autoregressive Image Generationcited by 7
  12. 2025DeepCompress: A Dual Reward Strategy for Dynamically Exploring and Compressing Reasoning Chainscited by 7
  13. 2026ORBIT: On-policy Exploration-Exploitation for Controllable Multi-Budget Reasoningcited by 7
  14. 2026Tackling Length Inflation Without Trade-offs: Group Relative Reward Rescaling for Reinforcement Learningcited by 7
  15. 2025Thinking Short and Right Over Thinking Long: Serving LLM Reasoning Efficiently and Accuratelycited by 6
  16. 2025REA-RL: Reflection-Aware Online Reinforcement Learning for Efficient Reasoningcited by 6
  17. 2025From Emergence to Control: Probing and Modulating Self-Reflection in Language Modelscited by 6
  18. 2026Early Stopping for Large Reasoning Models via Confidence Dynamicscited by 6
  19. 2026When Is Thinking Enough? Early Exit via Sufficiency Assessment for Efficient Reasoningcited by 6
  20. 2026FROST: Filtering Reasoning Outliers with Attention for Efficient Reasoningcited by 5
  21. 2025Beyond Token Length: Step Pruner for Efficient and Accurate Reasoning in Large Language Modelscited by 4
  22. 2026ConPress: Learning Efficient Reasoning from Multi-Question Contextual Pressurecited by 4
  23. 2026S3-CoT: Self-Sampled Succinct Reasoning Enables Efficient Chain-of-Thought LLMscited by 4
  24. 2026SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoningcited by 4
  25. 2025DRQA: Dynamic Reasoning Quota Allocation for Controlling Overthinking in Reasoning Large Language Modelscited by 3
  26. 2025Concise Reasoning in the Lens of Lagrangian Optimizationcited by 3
  27. 2025Incorporating Self-Rewriting into Large Language Model Reasoning Reinforcementcited by 3
  28. 2026SAW: Stage-Aware Dynamic Weighting for Multi-Objective Reinforcement Learning in Large Language Modelscited by 3
  29. 2026Contrastive On-Policy Distillationcited by 3
  30. 2025AdaThink-Med: Optimizing Inference-Time Compute for Medical Reasoning via Uncertainty Quantificationcited by 2
  31. 2025Adaptive Dual Reasoner: Large Reasoning Models Can Think Efficiently by Hybrid Reasoningcited by 2
  32. 2025Curriculum Design for Trajectory-Constrained Agent: Compressing Chain-of-Thought Tokens in LLMscited by 2
  33. 2025Efficient Reasoning via Thought-Training and Thought-Free Inferencecited by 2
  34. 2025Think Before You Prune: Selective Self-Generated Calibration for Pruning Large Reasoning Modelscited by 2
  35. 2026BFS-PO: Best-First Search for Large Reasoning Modelscited by 2
  36. 2026TRiMS: Real-Time Tracking of Minimal Sufficient Length for Efficient Reasoning via RLcited by 2
  37. 2026Does Accuracy Equal Evidence? Reasoning Faithfulness under KV Cache Compressioncited by 2
  38. 2025ORION: Teaching Language Models to Reason Efficiently in the Language of Thoughtcited by 1

50 most cited of 72 All 72 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Shorten After You're Right: Lazy Length Penalties for Reasoning RL
  2. 2026The Art of Efficient Reasoning: Data, Reward, and Optimization
  3. 2025SmartThinker: Learning to Compress and Preserve Reasoning by Step-Level Length Control
  4. 2025Marco-o1 v2: Towards Widening The Distillation Bottleneck for Reasoning Models
  5. 2025Shorter but not Worse: Frugal Reasoning via Easy Samples as Length Regularizers in Math RLVR
  6. 2025Adaptive Deep Reasoning: Triggering Deep Thinking When Needed
  7. 2025VeriThinker: Learning to Verify Makes Reasoning Model Efficient
  8. 2026Correct, Concise and Complete: Multi-stage Training For Adaptive Reasoning