# Walk Before You Run! Concise LLM Reasoning via Reinforcement Learning

arXiv: https://arxiv.org/abs/2505.21178 · HTML: https://mlsanity.com/paper/2505.21178/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 27 May 2025.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 11 of 11)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://mlsanity.com/paper/2503.20783/) (2025)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Process Reinforcement through Implicit Rewards](https://mlsanity.com/paper/2502.01456/) (2025)
- [Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond](https://mlsanity.com/paper/2503.10460/) (2025)
- [Concise Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2504.05185/) (2025)

## Built on by (most cited first, 16 of 16)

- [Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models](https://mlsanity.com/paper/2508.02120/) (2025)
- [OptimalThinkingBench: Evaluating Over and Underthinking in LLMs](https://mlsanity.com/paper/2508.13141/) (2025)
- [OckBench: Measuring the Efficiency of LLM Reasoning](https://mlsanity.com/paper/2511.05722/) (2025)
- [Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Models](https://mlsanity.com/paper/2601.11340/) (2026)
- [Think Dense, Not Long: Dynamic Decoupled Conditional Advantage for Efficient Reasoning](https://mlsanity.com/paper/2602.02099/) (2026)
- [Logit Arithmetic Elicits Long Reasoning Capabilities Without Training](https://mlsanity.com/paper/2510.09354/) (2025)
- [Not All Turns Are Equally Hard: Adaptive Thinking Budgets For Efficient Multi-Turn Reasoning in Agents](https://mlsanity.com/paper/2604.05164/) (2026)
- [Concise Reasoning in the Lens of Lagrangian Optimization](https://mlsanity.com/paper/2510.10168/) (2025)
- [Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning](https://mlsanity.com/paper/2603.00296/) (2026)
- [Draft-Thinking: Learning Efficient Reasoning in Long Chain-of-Thought LLMs](https://mlsanity.com/paper/2603.00578/) (2026)
- [Logit Arithmetic Elicits Long Reasoning Capabilities Without Training](https://mlsanity.com/paper/2507.12759/) (2025)
- [ThinkBrake: Efficient Reasoning via Log-Probability Margin Guided Decoding](https://mlsanity.com/paper/2510.00546/) (2025)
- [SPEAR: Distilling Domain-Adaptive Reasoning Skeletons via Sequential Symbolic Alignment in Reinforcement Learning](https://mlsanity.com/paper/2608.26550/) (2026)
- [When Less is Enough: Efficient Inference via Collaborative Reasoning](https://mlsanity.com/paper/2605.01111/) (2026)
- [Token-Operations-Oriented Inference Optimization Techniques for Large Models](https://mlsanity.com/paper/2606.20295/) (2026)
- [Intern-S2-Mobius: Foundation Model with Decoupled Knowledge and Reasoning](https://mlsanity.com/paper/2608.14290/) (2026)

## Abstract (arXiv)

As test-time scaling becomes a pivotal research frontier in Large Language Models (LLMs) development, contemporary and advanced post-training methodologies increasingly focus on extending the generation length of long Chain-of-Thought (CoT) responses to enhance reasoning capabilities toward DeepSeek R1-like performance. However, recent studies reveal a persistent overthinking phenomenon in state-of-the-art reasoning models, manifesting as excessive redundancy or repetitive thinking patterns in long CoT responses. To address this issue, in this paper, we propose a simple yet effective two-stage reinforcement learning framework for achieving concise reasoning in LLMs, named ConciseR. Specifically, the first stage, using more training steps, aims to incentivize the model's reasoning capabilities via Group Relative Policy Optimization with clip-higher and dynamic sampling components (GRPO++), and the second stage, using fewer training steps, explicitly enforces conciseness and improves efficiency via Length-aware Group Relative Policy Optimization (L-GRPO). Significantly, ConciseR only optimizes response length once all rollouts of a sample are correct, following the "walk before you run" principle. Extensive experimental results demonstrate that our ConciseR model, which generates more concise CoT reasoning responses, outperforms recent state-of-the-art reasoning models with zero RL paradigm across AIME 2024, MATH-500, AMC 2023, Minerva, and Olympiad benchmarks.