# Think Dense, Not Long: Dynamic Decoupled Conditional Advantage for Efficient Reasoning

arXiv: https://arxiv.org/abs/2602.02099 · HTML: https://mlsanity.com/paper/2602.02099/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 22 references that are in the map.
- First posted on arXiv 2 Feb 2026.

## Builds on (most cited first, 18 of 18)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [LIMO: Less is More for Reasoning](https://mlsanity.com/paper/2502.03387/) (2025)
- [O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning](https://mlsanity.com/paper/2501.12570/) (2025)
- [Training Language Models to Reason Efficiently](https://mlsanity.com/paper/2502.04463/) (2025)
- [Revisiting Demonstration Selection Strategies in In-Context Learning](https://mlsanity.com/paper/2401.12087/) (2024)
- [Thinking Fast and Right: Balancing Accuracy and Reasoning Length with Adaptive Rewards](https://mlsanity.com/paper/2505.18298/) (2025)
- [Revisiting Overthinking in Long Chain-of-Thought from the Perspective of Self-Doubt](https://mlsanity.com/paper/2505.23480/) (2025)
- [Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey](https://mlsanity.com/paper/2507.09662/) (2025)
- [IPO: Your Language Model is Secretly a Preference Classifier](https://mlsanity.com/paper/2502.16182/) (2025)
- [Walk Before You Run! Concise LLM Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2505.21178/) (2025)
- [Robust Knowledge Editing via Explicit Reasoning Chains for Distractor-Resilient Multi-Hop QA](https://mlsanity.com/paper/2509.01468/) (2025)

## Built on by (most cited first, 5 of 5)

- [Diagnosing and Mitigating Thinking Collapse in On-Policy Self-Distillation](https://mlsanity.com/paper/2607.10805/) (2026)
- [Context Language Models](https://mlsanity.com/paper/2609.37725/) (2026)
- [Beyond One-Size-Fits-All: Diagnosis-Driven Online Reinforcement Learning with Offline Priors](https://mlsanity.com/paper/2606.25527/) (2026)
- [Giving Credit Where It's Due: Redundancy-Aware Learning for Efficient Reasoning](https://mlsanity.com/paper/2609.27156/) (2026)
- [Quality Determines Direction, Length Shapes Magnitude: Length Control for Open-Ended Reinforcement Learning](https://mlsanity.com/paper/2609.34718/) (2026)

## Abstract (arXiv)

Reinforcement Learning with Verifiable Rewards (RLVR) can elicit strong multi-step reasoning, yet it often encourages overly verbose traces. Moreover, naive length penalties in group-relative optimization can severely hurt accuracy. We attribute this failure to two structural issues: (i) Dilution of Length Baseline, where incorrect responses (with zero length reward) depress the group baseline and over-penalize correct solutions; and (ii) Difficulty-Penalty Mismatch, where a static penalty cannot adapt to problem difficulty, suppressing necessary reasoning on hard instances while leaving redundancy on easy ones. We propose Dynamic Decoupled Conditional Advantage (DDCA) to decouple efficiency optimization from correctness. DDCA computes length advantages conditionally within the correct-response cluster to eliminate baseline dilution, and dynamically scales the penalty strength using the group pass rate as a proxy for difficulty. Experiments on GSM8K, MATH500, AMC23, and AIME25 show that DDCA consistently improves the efficiency--accuracy trade-off relative to adaptive baselines, reducing generated tokens by approximately 60% on simpler tasks (e.g., GSM8K) versus over 20% on harder benchmarks (e.g., AIME25), thereby maintaining or improving accuracy. Code is available at https://github.com/alphadl/DDCA.