mlsanityData to 7 Oct 2026

Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domains

arXiv 2605.18261 ↗First posted 18 May 2026cs.CLcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) has demonstrated promising potential to enhance the reasoning capabilities of large language models (LLMs) in domains such as mathematics and coding. However, its applications on knowledge-intensive domains have not been effectively explored due to the scarcity of high-quality verifiable data. Furthermore, current RLVR focuses solely on the correctness of final answers, leading to the limitations of flawed reasoning and sparse reward signals. In this work, we propose Knowledge-to-Verification (K2V), a framework that extends RLVR to knowledge-intensive domains through automated verifiable data synthesis, while enabling verification of the LLM's reasoning process. Extensive experiments demonstrate that K2V enhances the reasoning of LLM in knowledge-intensive domains without significantly compromising the model's general capabilities. This study also suggests that integrating automated data synthesis with reasoning verification is a promising direction to enhance model capabilities in these broader domains. Code is available at https://github.com/SeedScientist/K2V.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 36 of its 68 references that are in the map. First posted on arXiv 18 May 2026.

Builds on

36 of 68 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Klear-Reasoner: Advancing Reasoning Capability via Gradient-Preserving Clipping Policy Optimization11 Aug
  2. 2025OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling25 Jun
  3. 2025General-Reasoner: Advancing LLM Reasoning Across All Domains20 May
  4. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  5. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  6. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  7. 2025Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search4 Feb
  8. 2023Let's Verify Step by Step31 May

Other references · 28

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2024Qwen2.5 Technical Reportcited by 4,713
  7. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  10. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  11. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  12. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
16 more
  1. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  2. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  3. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  4. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
  5. 2025Reasoning Models Don't Always Say What They Thinkcited by 388
  6. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyondcited by 175
  7. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 121
  8. 2023DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Servicescited by 117
  9. 2022BERN2: an advanced neural biomedical named entity recognition and normalization toolcited by 42
  10. 2024Genie: Achieving Human Parity in Content-Grounded Datasets Generationcited by 31
  11. 2025CoT-Self-Instruct: Building high-quality synthetic prompts for reasoning and non-reasoning taskscited by 30
  12. 2025KARMA: Leveraging Multi-Agent LLMs for Automated Knowledge Graph Enrichmentcited by 26
  13. 2023LIQUID: A Framework for List Question Answering Dataset Generationcited by 19
  14. 2025Bottom-up Domain-specific Superintelligence: A Reliable Knowledge Graph is What We Needcited by 12
  15. 2025Rewarding Graph Reasoning Process makes LLMs more Generalized Reasonerscited by 10
  16. 2025SeedBench: A Multi-task Benchmark for Evaluating Large Language Models in Seed Sciencecited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
  2. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs
  3. 2025Extending RLVR to Open-Ended Tasks via Verifiable Multiple-Choice Reformulation
  4. 2025From Trial-and-Error to Improvement: A Systematic Analysis of LLM Exploration Mechanisms in RLVR
  5. 2026To Mix or To Merge: Toward Multi-Domain Reinforcement Learning for Large Language Models
  6. 2025RLPR: Extrapolating RLVR to General Domains without Verifiers
  7. 2025Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains
  8. 2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs