CLARity: Reasoning Consistency Alone Can Teach Reinforced Experts
On the map: LLM reasoning → Reinforcement learning for reasoning
Abstract (arXiv)
Training expert LLMs in domains with scarce data is difficult, often relying on multiple-choice questions (MCQs). However, standard outcome-based reinforcement learning (RL) on MCQs is risky. While it may improve accuracy, we observe it often degrades reasoning quality such as logical consistency. Existing solutions to supervise reasoning, such as large-scale Process Reward Models (PRMs), are prohibitively expensive. To address this, we propose CLARity, a cost-effective RL framework that enhances reasoning quality using only a small, general-purpose LLM. CLARity integrates a consistency-aware reward mechanism with a 2-stage refine-then-monitor training pipeline to enhance reasoning consistency, and a dynamic data reformulation strategy to to better exploit limited data. Experiments demonstrate that CLARity improves response consistency by 16.5% and accuracy by 7.5% over baselines. Human evaluations further confirm holistic improvements in coherence and professionalism. Thus, CLARity offers a generalizable solution that enables smaller models to effectively guide expert models by reasoning consistency. Our code is open sourced at: https://github.com/Infinite-set/CLARity
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 38 of its 49 references that are in the map. First posted on arXiv 10 Oct 2025.
Builds on
38 of 49 references.
Closest ancestors · 8
Same direction, newest first
- 2025ReCode: Reinforcing Code Generation with Reasoning-Process Rewards7 Aug
- 2025Truthful or Fabricated? Using Causal Attribution to Mitigate Reward Hacking in Explanations7 Apr
- 2025Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains31 Mar
- 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning20 Feb
- 2025LIMR: Less is More for RL Scaling17 Feb
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
- 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
- 2017Proximal Policy Optimization Algorithms20 Jul
Other references · 30
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2024GPT-4o System Cardcited by 4,467
- 2024A Survey on LLM-as-a-Judgecited by 1,384
- 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
- 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
- 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
- 2024OpenAI o1 System Cardcited by 1,220
- 2024DeepSeek-V3 Technical Reportcited by 724
- 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 446
- 2025Reasoning Models Don't Always Say What They Thinkcited by 388
- 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 377
18 more
- 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 338
- 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 322
- 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 257
- 2024Frontier Models are Capable of In-context Schemingcited by 252
- 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithfulcited by 166
- 2024Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoningcited by 108
- 2024Prover-Verifier Games improve legibility of LLM outputscited by 79
- 2025ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templatescited by 62
- 2024Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thoughtcited by 37
- 2025RAG-RL: Advancing Retrieval-Augmented Generation via RL and Curriculum Learningcited by 30
- 2024Towards Consistent Natural-Language Explanations via Explanation-Consistency Finetuningcited by 26
- 2025FineMedLM-o1: Enhancing Medical Knowledge Reasoning Ability of LLM from Supervised Fine-Tuning to Test-Time Trainingcited by 19
- 2025Reasoning-Table: Exploring Reinforcement Learning for Table Reasoningcited by 18
- 2024Chain-of-Probe: Examining the Necessity and Accuracy of CoT Step-by-Stepcited by 8
- 2025Does Learning Mathematical Problem-Solving Generalize to Broader Reasoning?cited by 7
- 2025CodeBoost: Boosting Code LLMs by Squeezing Knowledge from Code Snippets with RLcited by 7
- 2025Select to Know: An Internal-External Knowledge Self-Selection Framework for Domain-Specific Question Answeringcited by 3
- 2025From Graph Retrieval to Schema Realization: Counterfactual Validation for Text-to-SPARQL over Heterogeneous Knowledge Graphscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Beyond Correctness: Harmonizing Process and Outcome Rewards through RL Training
- 2025Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning
- 2024Teaching Large Language Models to Reason with Reinforcement Learning
- 2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
- 2025Reinforcement Learning vs. Distillation: Understanding Accuracy and Capability in LLM Reasoning
- 2025Beyond Correctness: Confidence-Aware Reward Modeling for Enhancing Large Language Model Reasoning
- 2025Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection
- 2025ConciseRL: Conciseness-Guided Reinforcement Learning for Efficient Reasoning Models