GeneralThinker: Domain-General Reasoning through Likelihood-Guided Answer-Conditioned Optimization
On the map: LLM reasoning → Reinforcement learning for reasoning
Abstract (arXiv)
Reinforcement learning with verifiable rewards improves language model reasoning, but its reliance on domain-specific verifiers, sparse outcome rewards, and coarse-grained credit assignment limits its applicability. We introduce GeneralThinker, an on-policy framework that reformulates reasoning supervision as dense answer-conditioned optimization, enabling response-level evaluation and token-level credit assignment without domain-specific verifiers. GeneralThinker evaluates generated reasoning trajectories using the likelihood of the ground-truth answer and derives token-wise compatibility signals for fine-grained credit assignment. To stabilize optimization, it constrains token-level updates through clipping and direction-preserving modulation. Across 11 benchmarks spanning mathematics, STEM, and general reasoning, GeneralThinker achieves the best average performance. Further analyses show that uncontrolled token-level modulation can destabilize training, whereas controlled modulation makes fine-grained credit assignment consistently effective.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 14 of its 33 references that are in the map. First posted on arXiv 27 May 2026.
Builds on
14 of 33 references.
Closest ancestors · 7
Same direction, newest first
- 2026Likelihood-Based Reward Designs for General LLM Reasoning3 Feb
- 2026Advancing General-Purpose Reasoning Models with Modular Gradient Surgery2 Feb
- 2026When Domains Interact: Asymmetric and Order-Sensitive Cross-Domain Effects in Reinforcement Learning for Reasoning1 Feb
- 2025Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models15 Dec
- 2025RLPR: Extrapolating RLVR to General Domains without Verifiers23 Jun
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
- 2017Proximal Policy Optimization Algorithms20 Jul
Other references · 7
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
- 2024OpenAI o1 System Cardcited by 1,244
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Reinforcing General Reasoning without Verifiers
- 2025General-Reasoner: Advancing LLM Reasoning Across All Domains
- 2025Token-Efficient RL for LLM Reasoning
- 2026DGPO: Distribution Guided Policy Optimization for Fine Grained Credit Assignment
- 2025GRPO-λ: Credit Assignment improves LLM Reasoning
- 2025Direct Reasoning Optimization: Token-Level Reasoning Reflectivity Meets Rubric Gates for Unverifiable Tasks
- 2025In-Token Rationality Optimization: Towards Accurate and Concise LLM Reasoning via Self-Feedback
- 2026DARL: Encouraging Diverse Answers for General Reasoning without Verifiers