mlsanityData to 14 Oct 2026

Klear-Reasoner: Advancing Reasoning Capability via Gradient-Preserving Clipping Policy Optimization

arXiv 2508.07629 ↗First posted 11 Aug 2025cs.LG cs.AI cs.CLcited by 39

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

We present Klear-Reasoner, a model with long reasoning capabilities that demonstrates careful deliberation during problem solving, achieving outstanding performance across multiple benchmarks. Although there are already many excellent works related to inference models in the current community, there are still many problems with reproducing high-performance inference models due to incomplete disclosure of training details. This report provides an in-depth analysis of the reasoning model, covering the entire post-training workflow from data preparation and long Chain-of-Thought supervised fine-tuning (long CoT SFT) to reinforcement learning (RL), along with detailed ablation studies for each experimental component. For SFT data, our experiments show that a small number of high-quality data sources are more effective than a large number of diverse data sources, and that difficult samples can achieve better results without accuracy filtering. In addition, we investigate two key issues with current clipping mechanisms in RL: Clipping suppresses critical exploration signals and ignores suboptimal trajectories. To address these challenges, we propose Gradient-Preserving clipping Policy Optimization (GPPO) that gently backpropagates gradients from clipped tokens. GPPO not only enhances the model's exploration capacity but also improves its efficiency in learning from negative samples. Klear-Reasoner exhibits exceptional reasoning abilities in mathematics and programming, scoring 90.5% on AIME 2024, 83.2% on AIME 2025, 66.0% on LiveCodeBench V5 and 58.1% on LiveCodeBench V6.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 28 references that are in the map. First posted on arXiv 11 Aug 2025.

Builds on

20 of 28 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Ring-lite: Scalable Reasoning via C3PO-Stabilized Reinforcement Learning for LLMs17 Jun
  2. 2025AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergy16 Jun
  3. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning30 May
  4. 2025AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning22 May
  5. 2025MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttraining12 May
  6. 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks7 Apr
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  9. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 11

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  4. 2024OpenAI o1 System Cardcited by 1,244
  5. 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 304
  6. 2025MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attentioncited by 263
  7. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyondcited by 175
  8. 2025Skywork Open Reasoner 1 Technical Reportcited by 165
  9. 2023TACO: Topics in Algorithmic COde generation datasetcited by 160
  10. 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Codingcited by 113
  11. 2025OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critiquecited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 28.

39 papers.

Sort:

By area:

  1. 2025The Path Not Taken: RLVR Provably Learns Off the Principalscited by 56
  2. 2025Prosperity before Collapse: How Far Can Off-Policy RL Reach with Stale Data on LLMs?cited by 53
  3. 2025Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Modelscited by 48
  4. 2025Agentic Entropy-Balanced Policy Optimizationcited by 40
  5. 2025Low-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Rewardcited by 25
  6. 2025CE-GPPO: Coordinating Entropy via Gradient-Preserving Clipping Policy Optimization in Reinforcement Learningcited by 23
  7. 2025When Importance Sampling Misallocates Credit: Asymmetric Ratios for Outcome-Supervised RLcited by 21
  8. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 15
  9. 2025Repurposing Synthetic Data for Fine-grained Search Agent Supervisioncited by 11
  10. 2026VESPO: Variational Sequence-Level Soft Policy Optimization for Stable Off-Policy LLM Trainingcited by 11
  11. 2026MASPO: Unifying Gradient Utilization, Probability Mass, and Signal Reliability for Robust and Sample-Efficient LLM Reasoningcited by 11
  12. 2026LLMs Can Learn to Reason Via Off-Policy RLcited by 10
27 more
  1. 2026AT2PO: Agentic Turn-based Policy Optimization via Tree Searchcited by 8
  2. 2026Rethinking Muon Beyond Pretraining: Spectral Failures and High-Pass Remedies for VLA and RLVRcited by 8
  3. 2025ScaleDiff: Scaling Difficult Problems for Advanced Mathematical Reasoningcited by 6
  4. 2026Ratio-Variance Regularized Policy Optimization for Efficient LLM Fine-tuningcited by 6
  5. 2026RAPO: Expanding Exploration for LLM Agents via Retrieval-Augmented Policy Optimizationcited by 6
  6. 2026From log π to π: Taming Divergence in Soft Clipping via Bilateral Decoupled Decay of Probability Gradient Weightcited by 6
  7. 2026Orchestrating Tokens and Sequences: Dynamic Hybrid Policy Optimization for RLVRcited by 5
  8. 2026Missing Old Logits in Asynchronous Agentic RL: Semantic Mismatch and Repair Methods for Off-Policy Correctioncited by 5
  9. 2026Logics-STEM: Empowering LLM Reasoning via Failure-Driven Post-Training and Document Knowledge Enhancementcited by 4
  10. 2026APPO: Agentic Procedural Policy Optimizationcited by 4
  11. 2026QUATRO: Query-Adaptive Trust Region Policy Optimization for LLM Fine-tuningcited by 3
  12. 2026Beyond Euclidean Clipping: Overcoming Exploration Collapse in LLM RL via Riemannian Isometric Policy Optimizationcited by 3
  13. 2025Entropy Ratio Clipping as a Soft Global Constraint for Stable Reinforcement Learningcited by 2
  14. 2026Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learningcited by 2
  15. 2026X-Coder: Advancing Competitive Programming with Synthetic Tasks, Solutions, and Testscited by 1
  16. 2026D-VLA: A High-Concurrency Distributed Asynchronous Reinforcement Learning Framework for Vision-Language-Action Modelscited by 1
  17. 2026RLVR Datasets and Where to Find Them: Tracing Data Lineage for Better Training Datacited by 1
  18. 2026BASIS: Batchwise Advantage Estimation from Single-Rollout Information Sharing for LLM Reasoningcited by 1
  19. 2026ACRL: Adaptive Control of Training-Inference Discrepancy for Stable Reinforcement Learningcited by 1
  20. 2026Primal Generation, Dual Judgment: Self-Training from Test-Time Scalingcited by 0
  21. 2026Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domainscited by 0
  22. 2026Ratio-Variance Regularized Policy Optimizationcited by 0
  23. 2026What are Key Factors for Updates in RL for LLM Reasoning?cited by 0
  24. 2026UP: Unbounded Positive Asymmetric Optimization for Breaking the Exploration-Stability Dilemmacited by 0
  25. 2026Lightning Weave: Improving the Accuracy-Efficiency Frontier of Reasoning Models through Capability Compositioncited by 0
  26. 2026GRRR: The Geometry of Reshaping, Rotation, and Routing in Decoder LLM post-trainingcited by 0
  27. 2026CATCH: A Controllable Analysis Testbed for Reward Hacking in Coding RLcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026On-Policy Supervised Fine-Tuning for Efficient Reasoning
  2. 2025SmartThinker: Learning to Compress and Preserve Reasoning by Step-Level Length Control
  3. 2024RL-STaR: Theoretical Analysis of Reinforcement Learning Frameworks for Self-Taught Reasoner
  4. 2026ConMax: Confidence-Maximizing Compression for Efficient Chain-of-Thought Reasoning
  5. 2025Reasoning Vectors: Transferring Chain-of-Thought Capabilities via Task Arithmetic
  6. 2025ConciseRL: Conciseness-Guided Reinforcement Learning for Efficient Reasoning Models
  7. 2026Drop the Act: Probe-Filtered RL for Faithful Chain-of-Thought Reasoning
  8. 2025How Far Are We from Optimal Reasoning Efficiency?