# ReCrit: Transition-Aware Reinforcement Learning for Scientific Critic Reasoning

arXiv: https://arxiv.org/abs/2605.18799 · HTML: https://mlsanity.com/paper/2605.18799/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 38 of its 50 references that are in the map.
- First posted on arXiv 11 May 2026.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 30 of 38)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025)
- [Qwen3.5-Omni Technical Report](https://mlsanity.com/paper/2604.15804/) (2026)
- [RM-R1: Reward Modeling as Reasoning](https://mlsanity.com/paper/2505.02387/) (2025)
- [ELEPHANT: Measuring and understanding social sycophancy in LLMs](https://mlsanity.com/paper/2505.13995/) (2025)
- [Measuring Sycophancy of Language Models in Multi-turn Dialogues](https://mlsanity.com/paper/2505.23840/) (2025)
- [Reinforcement Learning for LLM Post-Training: A Survey](https://mlsanity.com/paper/2407.16216/) (2024)
- [Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback](https://mlsanity.com/paper/2506.03106/) (2025)
- [Reinforcing Multi-Turn Reasoning in LLM Agents via Fine-Grained Reward Structure and Credit Assignment](https://mlsanity.com/paper/2505.11821/) (2025)
- [LLM-based Multi-Agent Reinforcement Learning: Current and Future Directions](https://mlsanity.com/paper/2405.11106/) (2024)
- [Intern-S1: A Scientific Multimodal Foundation Model](https://mlsanity.com/paper/2508.15763/) (2025)
- [SciMaster: Towards General-Purpose Scientific AI Agents, Part I. X-Master as Foundation: Can We Lead on Humanity's Last Exam?](https://mlsanity.com/paper/2507.05241/) (2025)
- [SPC: Evolving Self-Play Critic via Adversarial Games for LLM Reasoning](https://mlsanity.com/paper/2504.19162/) (2025)
- [AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Training](https://mlsanity.com/paper/2507.01663/) (2025)
- [A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers](https://mlsanity.com/paper/2508.21148/) (2025)
- [RollPacker: Mitigating Long-Tail Rollouts for Fast, Synchronous RL Post-Training](https://mlsanity.com/paper/2509.21009/) (2025)
- [TRUTH DECAY: Quantifying Multi-Turn Sycophancy in Language Models](https://mlsanity.com/paper/2503.11656/) (2025)
- [A Survey on Multi-Turn Interaction Capabilities of Large Language Models](https://mlsanity.com/paper/2501.09959/) (2025)
- [OpenDataLab: Empowering General Artificial Intelligence with Open Datasets](https://mlsanity.com/paper/2407.13773/) (2024)
- [Intern-S1-Pro: Scientific Multimodal Foundation Model at Trillion Scale](https://mlsanity.com/paper/2603.25040/) (2026)
- [StepWiser: Stepwise Generative Judges for Wiser Reasoning](https://mlsanity.com/paper/2508.19229/) (2025)
- [Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows](https://mlsanity.com/paper/2512.16969/) (2025)
- [BrokenMath: A Benchmark for Sycophancy in Theorem Proving with LLMs](https://mlsanity.com/paper/2510.04721/) (2025)
- [ClimaQA: An Automated Evaluation Framework for Climate Question Answering Models](https://mlsanity.com/paper/2410.16701/) (2024)
- [ProRL Agent: Rollout-as-a-Service for RL Training of Multi-Turn LLM Agents](https://mlsanity.com/paper/2603.18815/) (2026)

## Abstract (arXiv)

Large language models can fail in critic interaction not only by answering incorrectly, but also by abandoning an initially correct scientific solution after user criticism. This is especially risky in scientific reasoning, where user criticism can turn a valid answer into an incorrect one. We frame critic interaction as an inter-turn correctness-transition problem rather than a final-answer accuracy problem, and identify three challenges: transition awareness, decoupling useful correction from harmful sycophancy, and scalable rollout. We propose ReCrit, a transition-aware reinforcement learning framework that decomposes Initial-to-Critic behavior into four quadrants: Correction, Sycophancy, Robustness, and Boundary. ReCrit rewards correction and robustness, penalizes sycophancy, and treats persistent errors as weak boundary signals. To make interaction training practical, ReCrit further uses dynamic asynchronous rollout with tail-adaptive completion to reduce rollout waiting. On three scientific reasoning benchmarks, ChemBench, TRQA, and EarthSE, ReCrit improves average Critic accuracy from 38.15 to 51.49 on Qwen3.5-4B and from 45.40 to 55.59 on Qwen3.5-9B. Ablations show that final-answer rewards provide little interaction-level gain, while transition-aware rewards and quadrant weighting produce more distinguishable training signals and larger net Critic-stage improvement. The code is available at https://github.com/black-yt/ReCrit .