mlsanityData to 7 Oct 2026

QLPO: Quadrant-weighted Sampling for Length-aware Policy Optimization

arXiv 2607.21793 ↗First posted 23 Jul 2026cs.AIcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Recent large reasoning models often develop long chain-of-thought responses during reinforcement learning (RL), resulting in high inference latency and deployment cost. Existing methods for response length control typically rely on explicit length penalties or additional control modules, which require careful tuning and may compromise reasoning quality. We propose Quadrant-weighted Sampling for Length-aware Policy Optimization (QLPO), a simple resampling-based variant of GRPO that introduces implicit length control without modifying the reward function. QLPO first over-generates candidate responses and then resamples the training group by preserving the empirical correct/incorrect ratio while favoring short correct responses and long incorrect responses. This reshapes the training distribution and implicitly encourages shorter model outputs. Across models ranging from 1.5B to 32B parameters, including both base models and strong reasoning models, QLPO consistently improves the accuracy-length trade-off. It reduces response length by 30% to 70% while preserving reasoning performance. These results suggest that structured resampling provides an effective and robust approach to efficient reasoning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 43 of its 52 references that are in the map. First posted on arXiv 23 Jul 2026.

Builds on

43 of 52 references.

Closest ancestors · 18

Same direction, newest first

  1. 2026Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens13 Feb
  2. 2026Stop Unnecessary Reflection: Training LRMs for Efficient Reasoning with Adaptive Reflection and Length Coordinated Penalty12 Feb
  3. 2026Correct, Concise and Complete: Multi-stage Training For Adaptive Reasoning6 Jan
  4. 2025Leash: Adaptive Length Penalty and Reward Shaping for Efficient Large Reasoning Model25 Dec
  5. 2025DeepCompress: A Dual Reward Strategy for Dynamically Exploring and Compressing Reasoning Chains31 Oct
  6. 2025DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learning16 Oct
  7. 2025Beyond Token Length: Step Pruner for Efficient and Accurate Reasoning in Large Language Models4 Oct
  8. 2025Reconsidering Overthinking: Penalizing Internal and External Redundancy in CoT Reasoning4 Aug
  9. 2025SmartThinker: Learning to Compress and Preserve Reasoning by Step-Level Length Control6 Jul
  10. 2025Just Enough Thinking: Efficient Reasoning with Adaptive Length Penalties Reinforcement Learning5 Jun
  11. 2025Thinking Fast and Right: Balancing Accuracy and Reasoning Length with Adaptive Rewards23 May
  12. 2025Learn to Reason Efficiently with Adaptive Length-based Reward Shaping21 May
6 more
  1. 2025Thinkless: LLM Learns When to Think19 May
  2. 2025AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learning17 May
  3. 2025Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RL16 May
  4. 2025ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning2 Apr
  5. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning6 Mar
  6. 2025DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models6 Mar

Other references · 25

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2025Qwen2.5-VL Technical Reportcited by 5,418
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2023Let's Verify Step by Stepcited by 3,598
  8. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  10. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  11. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  12. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
13 more
  1. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  2. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  3. 2025Process Reinforcement through Implicit Rewardscited by 478
  4. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 377
  5. 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 376
  6. 2023A Long Way to Go: Investigating Length Correlations in RLHFcited by 285
  7. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  8. 2025Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoningcited by 75
  9. 2025ConciseRL: Conciseness-Guided Reinforcement Learning for Efficient Reasoning Modelscited by 14
  10. 2025LSPO: Length-aware Dynamic Sampling for Policy Optimization in LLM Reasoningcited by 9
  11. 2025RoRecomp: Enhancing Reasoning Efficiency via Rollout Response Recomposition in Reinforcement Learningcited by 3
  12. 2026LACONIC: Length-Aware Constrained Reinforcement Learning for LLMcited by 3
  13. 2026DARTS: Distribution-Aware Active Rollout Trajectory Shaping for Accelerating LLM Reinforcement Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Balancing the Reasoning Load: Difficulty-Differentiated Policy Optimization with Length Redistribution for Efficient and Robust Reinforcement Learning
  2. 2025LAPO: Internalizing Reasoning Efficiency via Length-Adaptive Policy Optimization
  3. 2026Length-Unbiased Sequence Policy Optimization: Revealing and Controlling Response Length Variation in RLVR
  4. 2026Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning
  5. 2025ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning
  6. 2025AALC: Large Language Model Efficient Reasoning via Adaptive Accuracy-Length Control
  7. 2025DRPO: Efficient Reasoning via Decoupled Reward Policy Optimization
  8. 2026On the Optimal Reasoning Length for RL-Trained Language Models