# Boosting LLM Reasoning via Human-Inspired Reward Shaping

arXiv: https://arxiv.org/abs/2602.04265 · HTML: https://mlsanity.com/paper/2602.04265/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 24 of its 45 references that are in the map.
- First posted on arXiv 4 Feb 2026.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 24 of 24)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement](https://mlsanity.com/paper/2409.12122/) (2024)
- [Group Sequence Policy Optimization](https://mlsanity.com/paper/2507.18071/) (2025)
- [Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models](https://mlsanity.com/paper/2503.16419/) (2025)
- [Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models](https://mlsanity.com/paper/2312.06585/) (2023)
- [ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning](https://mlsanity.com/paper/2504.01296/) (2025)
- [Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Models](https://mlsanity.com/paper/2508.10751/) (2025)
- [Learn to Reason Efficiently with Adaptive Length-based Reward Shaping](https://mlsanity.com/paper/2505.15612/) (2025)
- [Just Enough Thinking: Efficient Reasoning with Adaptive Length Penalties Reinforcement Learning](https://mlsanity.com/paper/2506.05256/) (2025)
- [Your Group-Relative Advantage Is Biased](https://mlsanity.com/paper/2601.08521/) (2026)
- [Rethinking Entropy Regularization in Large Reasoning Models](https://mlsanity.com/paper/2509.25133/) (2025)
- [Train Long, Think Short: Curriculum Learning for Efficient Reasoning](https://mlsanity.com/paper/2508.08940/) (2025)
- [GTPO and GRPO-S: Token and Sequence-Level Reward Shaping with Policy Entropy](https://mlsanity.com/paper/2508.04349/) (2025)
- [CURE: Critical-Token-Guided Re-Concatenation for Entropy-Collapse Prevention](https://mlsanity.com/paper/2508.11016/) (2025)
- [Beyond the Sampled Token: Preserving Candidate Support in RLVR](https://mlsanity.com/paper/2510.14807/) (2025)
- [The Best of N Worlds: Aligning Reinforcement Learning with Best-of-N Sampling via max@k Optimisation](https://mlsanity.com/paper/2510.23393/) (2025)
- [Risk-Sensitive RL for Alleviating Exploration Dilemmas in Large Language Models](https://mlsanity.com/paper/2509.24261/) (2025)
- [EntroPIC: Towards Stable Long-Term Training of LLMs via Entropy Stabilization with Proportional-Integral Control](https://mlsanity.com/paper/2511.15248/) (2025)
- [RSPO: Risk-Seeking Policy Optimization for Pass@k and Max@k Metrics in Large Language Models](https://mlsanity.com/paper/2508.01174/) (2025)

## Built on by (most cited first, 1 of 1)

- [Rethinking the Comparison Unit in Sequence-Level Reinforcement Learning: An Equal-Length Paired Training Framework from Loss Correction to Sample Construction](https://mlsanity.com/paper/2604.17328/) (2026)

## Abstract (arXiv)

Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a promising paradigm for enhancing reasoning in Large Language Models (LLMs). However, existing reward formulations typically treat exploration and consolidation as a monolithic process, resulting in entangled stage-wise learning dynamics. This contradicts the natural learning behavior of human learners. In human learning, individuals adopt distinct behavioral patterns toward mastered versus unfamiliar problems. When confronting unmastered challenges, humans prioritize broad exploration to seek viable solutions. By contrast, for well-mastered problems, they focus instead on reasoning condensation and knowledge abstraction to distill concise underlying principles. Motivated by this gap, we introduce T2T(Thickening-to-Thinning), a dynamic reward framework inspired by human learning processes. Specifically, it implements a dual-phase mechanism: (1) On incorrect attempts, T2T incentivizes "thickening" to broaden the search space and explore novel solution paths; (2) Upon achieving correctness, it shifts to "thinning", imposing length penalties to discourage redundancy, thereby fostering model confidence and crystallizing reasoning capabilities. Extensive experiments on mathematical benchmarks (MATH-500, AIME, AMC) across 5 mainstream LLMs demonstrate that T2T significantly outperforms standard GRPO and recent baselines, achieving superior performance.