mlsanityData to 7 Oct 2026

Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains

arXiv 2503.23829 ↗First posted 31 Mar 2025cs.CLcited by 150

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) has demonstrated significant success in enhancing mathematical reasoning and coding performance of large language models (LLMs), especially when structured reference answers are accessible for verification. However, its extension to broader, less structured domains remains unexplored. In this work, we investigate the effectiveness and scalability of RLVR across diverse real-world domains including medicine, chemistry, psychology, economics, and education, where structured reference answers are typically unavailable. We reveal that binary verification judgments on broad-domain tasks exhibit high consistency across various LLMs provided expert-written reference answers exist. Motivated by this finding, we utilize a generative scoring technique that yields soft, model-based reward signals to overcome limitations posed by binary verifications, especially in free-form, unstructured answer scenarios. We further demonstrate the feasibility of training cross-domain generative reward models using relatively small (7B) LLMs without the need for extensive domain-specific annotation. Through comprehensive experiments, our RLVR framework establishes clear performance gains, significantly outperforming state-of-the-art open-source aligned models such as Qwen2.5-72B and DeepSeek-R1-Distill-Qwen-32B across domains in free-form settings. Our approach notably enhances the robustness, flexibility, and scalability of RLVR, representing a substantial step towards practical reinforcement learning applications in complex, noisy-label scenarios.

Key facts. Cited by 150 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 40 references that are in the map. First posted on arXiv 31 Mar 2025.

Builds on

30 of 40 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  3. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning20 Feb
  4. 2025S2R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning18 Feb
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  6. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
  7. 2024OpenRFT: Adapting Reasoning Foundation Model for Domain-specific Tasks with Reinforcement Fine-Tuning22 Dec
  8. 2024Generative Verifiers: Reward Modeling as Next-Token Prediction27 Aug
  9. 2023Let's Verify Step by Step31 May

Other references · 21

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2021Program Synthesis with Large Language Modelscited by 3,509
  8. 2020Learning to summarize from human feedbackcited by 2,562
  9. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  10. 2021Measuring Coding Challenge Competence With APPScited by 879
  11. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  12. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 257
9 more
  1. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 236
  2. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 146
  3. 2024MAmmoTH2: Scaling Instructions from the Webcited by 143
  4. 2024Stream of Search (SoS): Learning to Search in Languagecited by 141
  5. 2024Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizingcited by 123
  6. 2025NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questionscited by 81
  7. 2025SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuningcited by 27
  8. 2024How Do Humans Write Code? Large Models Do It the Same Way Toocited by 11
  9. 2021Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Datacited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 84; 2026: 66.

150 papers.

Sort:

By area:

  1. 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domainscited by 328
  2. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 210
  3. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  4. 2025General-Reasoner: Advancing LLM Reasoning Across All Domainscited by 132
  5. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 121
  6. 2025Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoningcited by 101
  7. 2025Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Librarycited by 94
  8. 2025A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibilitycited by 89
  9. 2025Reinforcing General Reasoning without Verifierscited by 76
  10. 2025One Token to Fool LLM-as-a-Judgecited by 56
  11. 2026Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Taskscited by 55
  12. 2025ExGRPO: Learning to Reason from Experiencecited by 49
38 more
  1. 2025Writing-Zero: Bridge the Gap Between Non-verifiable Tasks and Verifiable Rewardscited by 47
  2. 2025Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoningcited by 43
  3. 2025s3: You Don't Need That Much Data to Train a Search Agent via RLcited by 41
  4. 2025CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Rewardcited by 39
  5. 2025Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewardscited by 38
  6. 2025CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learningcited by 36
  7. 2025Unlocking Exploration in RLVR: Uncertainty-aware Advantage Shaping for Deeper Reasoningcited by 34
  8. 2025Guided Self-Evolving LLMs with Minimal Human Supervisioncited by 33
  9. 2026Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 32
  10. 2025Energy-Based Transformers are Scalable Learners and Thinkerscited by 30
  11. 2025Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoningcited by 29
  12. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 29
  13. 2025Reward Is Enough: LLMs Are In-Context Reinforcement Learnerscited by 29
  14. 2025Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Modelscited by 28
  15. 2025QA-LIGN: Aligning LLMs through Constitutionally Decomposed QAcited by 28
  16. 2025CAPO: Towards Enhancing LLM Reasoning through Generative Credit Assignmentcited by 25
  17. 2025RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agentscited by 24
  18. 2026TIPS: Turn-Level Information-Potential Reward Shaping for Search-Augmented LLMscited by 24
  19. 2026Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learningcited by 23
  20. 2025Outcome-based Reinforcement Learning to Predict the Futurecited by 21
  21. 2025Sotopia-RL: Reward Design for Social Intelligencecited by 20
  22. 2025VerifyBench: Benchmarking Reference-based Reward Systems for Large Language Modelscited by 19
  23. 2025From Accuracy to Robustness: A Study of Rule- and Model-based Verifiers in Mathematical Reasoningcited by 19
  24. 2025Shop-R1: Rewarding LLMs to Simulate Human Behavior in Online Shopping via Reinforcement Learningcited by 19
  25. 2025Hybrid Reinforcement: When Reward Is Sparse, It's Better to Be Densecited by 19
  26. 2025OmniAD: Detect and Understand Industrial Anomaly via Multimodal Reasoningcited by 18
  27. 2026Learning to Self-Verify Makes Language Models Better Reasonerscited by 18
  28. 2025Direct Reasoning Optimization: Token-Level Reasoning Reflectivity Meets Rubric Gates for Unverifiable Taskscited by 16
  29. 2025NOVER: Incentive Training for Language Models via Verifier-Free Reinforcement Learningcited by 15
  30. 2025Xiaomi MiMo-VL-Miloco Technical Reportcited by 15
  31. 2026Benchmarking Reward Hack Detection in Code Environments via Contrastive Analysiscited by 15
  32. 2025Training LLMs for EHR-Based Reasoning Tasks via Reinforcement Learningcited by 14
  33. 2025Loong: Synthesize Long Chain-of-Thoughts at Scale through Verifierscited by 14
  34. 2025CTRLS: Chain-of-Thought Reasoning via Latent State-Transitioncited by 13
  35. 2025Environment Scaling for Interactive Agentic Experience Collection: A Surveycited by 13
  36. 2025Activation Control for Efficiently Eliciting Long Chain-of-thought Ability of Language Modelscited by 12
  37. 2025RECAST: Expanding the Boundaries of LLMs' Complex Instruction Following with Multi-Constraint Datacited by 11
  38. 2025Student-Centered Distillation Narrows the Agentic Gap Between Small and Large LLMscited by 11

50 most cited of 150 All 150 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025RLPR: Extrapolating RLVR to General Domains without Verifiers
  2. 2025Extending RLVR to Open-Ended Tasks via Verifiable Multiple-Choice Reformulation
  3. 2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
  4. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs
  5. 2026To Mix or To Merge: Toward Multi-Domain Reinforcement Learning for Large Language Models
  6. 2025Reinforcement Learning with Rubric Anchors
  7. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
  8. 2026Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domains