SHAPE: Stage-aware Hierarchical Advantage via Potential Estimation for LLM Reasoning
On the map: LLM reasoning → Mathematical reasoning
Abstract (arXiv)
Process supervision has emerged as a promising approach for enhancing LLM reasoning, yet existing methods fail to distinguish meaningful progress from mere verbosity, leading to limited reasoning capabilities and unresolved token inefficiency. To address this, we propose Stage-aware Hierarchical Advantage via Potential Estimation (SHAPE), a framework that formalizes reasoning as a trajectory through a state space of empirical solvability. SHAPE introduces a hierarchical credit assignment mechanism: at the segment level, it employs a stage-aware advantage function to prioritize efficient breakthroughs in low-potential states; at the token level, it utilizes entropy-driven redistribution to sharpen execution signals. Extensive experiments in math reasoning across three base models and five benchmarks demonstrate that SHAPE achieves an average accuracy gain of 3% with 30% reduced token consumption.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 24 of its 37 references that are in the map. First posted on arXiv 8 Apr 2026.
Builds on
24 of 37 references.
Closest ancestors · 1
Same direction, newest first
Other references · 23
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2025Qwen3 Technical Reportcited by 8,820
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
- 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
- 2024OpenAI o1 System Cardcited by 1,244
- 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
- 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
- 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 559
- 2025Process Reinforcement through Implicit Rewardscited by 504
11 more
- 2024VinePPO: Refining Credit Assignment in RL Training of LLMscited by 183
- 2024DPO Meets PPO: Reinforced Token Optimization for RLHFcited by 127
- 2025Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuningcited by 123
- 2025What's Behind PPO's Collapse in Long-CoT? Value Optimization Holds the Secretcited by 98
- 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
- 2025Segment Policy Optimization: Effective Segment-Level Credit Assignment in RL for Large Language Modelscited by 75
- 2025rStar2-Agent: Agentic Reasoning Technical Reportcited by 54
- 2025GTPO and GRPO-S: Token and Sequence-Level Reward Shaping with Policy Entropycited by 18
- 2025Exploring the System 1 Thinking Capability of Large Reasoning Modelscited by 10
- 2025VRPO: Rethinking Value Modeling for Robust RL under Noisy Supervision in LLM Post-Trainingcited by 9
- 2026ATTNPO: Attention-Guided Process Supervision for Efficient Reasoningcited by 9
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Unmasking Reasoning Processes: A Process-aware Benchmark for Evaluating Structural Mathematical Reasoning in LLMs
- 2026Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning
- 2024An Empirical Study of Data Ability Boundary in LLMs' Math Reasoning
- 2026LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals
- 2025Less is More Tokens: Efficient Math Reasoning via Difficulty-Aware Chain-of-Thought Distillation
- 2025Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedback
- 2026The Reasoning Tax: Token Economics of LLM Reasoning Across Task Types and Deployment Contexts
- 2026Math Reasoning in LLMs is Organized by Approach, Not Topic