mlsanityData to 14 Oct 2026

VerIF: Verification Engineering for Reinforcement Learning in Instruction Following

arXiv 2506.09942 ↗First posted 11 Jun 2025cs.CL cs.AIcited by 44

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) has become a key technique for enhancing large language models (LLMs), with verification engineering playing a central role. However, best practices for RL in instruction following remain underexplored. In this work, we explore the verification challenge in RL for instruction following and propose VerIF, a verification method that combines rule-based code verification with LLM-based verification from a large reasoning model (e.g., QwQ-32B). To support this approach, we construct a high-quality instruction-following dataset, VerInstruct, containing approximately 22,000 instances with associated verification signals. We apply RL training with VerIF to two models, achieving significant improvements across several representative instruction-following benchmarks. The trained models reach state-of-the-art performance among models of comparable size and generalize well to unseen constraints. We further observe that their general capabilities remain unaffected, suggesting that RL with VerIF can be integrated into existing RL recipes to enhance overall model performance. We have released our datasets, codes, and models to facilitate future research at https://github.com/THU-KEG/VerIF.

Key facts. Cited by 44 papers in the map (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 28 of its 48 references that are in the map. First posted on arXiv 11 Jun 2025.

Builds on

28 of 48 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training28 Jan

Other references · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  6. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  7. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  8. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 903
  9. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
  10. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  11. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  12. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
15 more
  1. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  2. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 392
  3. 2025ToolRL: Reward is All Tool Learning Needscited by 362
  4. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 265
  5. 2025Inference-Time Scaling for Generalist Reward Modelingcited by 239
  6. 2025Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learningcited by 161
  7. 2024Optimization-based Prompt Injection Attack to LLM-as-a-Judgecited by 143
  8. 2024Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Followingcited by 102
  9. 2025Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systemscited by 51
  10. 2024Why Do Large Language Models (LLMs) Struggle to Count Letters?cited by 38
  11. 2024IOPO: Empowering LLMs with Complex Instruction Following via Input-Output Preference Optimizationcited by 31
  12. 2025R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learningcited by 21
  13. 2025UltraIF: Advancing Instruction Following from the Wildcited by 18
  14. 2024Can We Count on LLMs? The Fixed-Effect Fallacy and Claims of GPT-4 Capabilitiescited by 13
  15. 2025Step-by-Step Mastery: Enhancing Soft Constraint Following Ability of Large Language Modelscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 35.

44 papers.

Sort:

By area:

  1. 2026Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunitiescited by 20
  2. 2026RubricEval: A Rubric-Level Meta-Evaluation Benchmark for LLM Judges in Instruction Followingcited by 20
  3. 2026Reproducing, Analyzing, and Detecting Reward Hacking in Rubric-Based Reinforcement Learningcited by 16
  4. 2026Chaining the Evidence: Robust Reinforcement Learning for Deep Search Agents with Citation-Aware Rubric Rewardscited by 14
  5. 2025AutoRubric: Rubric-Based Generative Rewards for Faithful Multimodal Reasoningcited by 11
  6. 2026IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluationcited by 10
  7. 2026Rubrics to Tokens: Bridging Response-level Rubrics and Token-level Rewards in Instruction Following Taskscited by 10
  8. 2026AgentIF-OneDay: A Task-level Instruction-Following Benchmark for General AI Agents in Daily Scenarioscited by 9
  9. 2025Replay Failures as Successes: Sample-Efficient Reinforcement Learning for Instruction Followingcited by 8
  10. 2025IFDECORATOR: Wrapping Instruction Following Reinforcement Learning with Verifiable Rewardscited by 7
  11. 2026Can LLM-as-a-Judge Reliably Verify Rubrics in Agentic Scenarios?cited by 7
  12. 2025Instructions are all you need: Self-supervised Reinforcement Learning for Instruction Followingcited by 6
32 more
  1. 2025Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Modelscited by 5
  2. 2025Reasoning Up the Instruction Ladder for Controllable Language Modelscited by 5
  3. 2025IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluationcited by 4
  4. 2025Beyond the Trade-off: Self-Supervised Reinforcement Learning for Reasoning Models' Instruction Followingcited by 3
  5. 2026Experience is the Best Teacher: Motivating Effective Exploration in Reinforcement Learning for LLMscited by 3
  6. 2026Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skillcited by 3
  7. 2026ComplexConstraints and Beyond: Expert Rubrics for RLVRcited by 3
  8. 2026On the Paradoxical Interference between Instruction-Following and Task Solvingcited by 2
  9. 2026CoRT: Counterfactual Replay for Token-Level Rubric-Guided Policy Optimizationcited by 2
  10. 2026Distill What You Trust: Reliability-Aware Multi-Teacher On-Policy Distillationcited by 2
  11. 2026Precision over Diversity: High-Precision Reward Generalizes to Robust Instruction Followingcited by 1
  12. 2026MM-THEBench: Do Reasoning MLLMs Think Reasonably?cited by 1
  13. 2026FMBench: Adaptive Large Language Model Output Formattingcited by 1
  14. 2026SEIF: Self-Evolving Reinforcement Learning for Instruction Followingcited by 1
  15. 2026Prompt-Level Reward Specifications for Open-Ended Post-Trainingcited by 1
  16. 2026Verifier-Induced Support Reshaping in On-Policy Optimizationcited by 1
  17. 2026WebGrader: Training LLMs for Web Development with Self-Evolving Programmatic Gradercited by 1
  18. 2026CARE: Contrastive Anchor-based Rubric Evolution for Large Language Model Post-Trainingcited by 1
  19. 2026From Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mixcited by 1
  20. 2025Financial Instruction Following Evaluation (FIFE)cited by 0
  21. 2026LsrIF: Enhancing Logic-Structured Instruction Following of Large Language Modelscited by 0
  22. 2026Trust Your Memory: Verifiable Control of Smart Homes through Reinforcement Learning with Multi-dimensional Rewardscited by 0
  23. 2026From Coarse to Fine: Benchmarking and Reward Modeling for Writing-Centric Generation Taskscited by 0
  24. 2026TinyJudge: Unverifiable Constraint Alignment via Lightweight Specialist Ensemblescited by 0
  25. 2026STAIF: A Stage-wise Optimization for Complex Instruction Followingcited by 0
  26. 2026Mitigating Exploration Bias in RL for Multi-Instruction Followingcited by 0
  27. 2026Cross-Relational Preference Learning for Better LLM Instruction Followingcited by 0
  28. 2026MM-IFEval-Pro: A Multilingual and Attack-Resistant Benchmark for Instruction-Following in Vision-Language Modelscited by 0
  29. 2026Towards Scalable RLVR: Multimodal Instruction Following Data Synthesis and Distillationcited by 0
  30. 2026Counterfactual Constraint-Conditioned On-Policy Distillation for Multi-Constraint Instruction Followingcited by 0
  31. 2026ScopeIF: Improving Scope-Aware Precise Instruction-Following in Large Language Models via Graded Reward Modelingcited by 0
  32. 2026Reflective Prompt Optimization of a Deep-Research Agent: A Topic-Disjoint, Multi-Seed Protocol for Measuring Transferable Gains and Prompt-Level Overfittingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025VerifyBench: A Systematic Benchmark for Evaluating Reasoning Verifiers Across Domains
  2. 2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
  3. 2025Extending RLVR to Open-Ended Tasks via Verifiable Multiple-Choice Reformulation
  4. 2026An Imperfect Verifier is Good Enough: Learning with Noisy Rewards
  5. 2026Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domains
  6. 2026SUPERNOVA: Eliciting General Reasoning in LLMs with Reinforcement Learning on Natural Instructions
  7. 2025Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewards
  8. 2026Delay, Plateau, or Collapse: Evaluating the Impact of Systematic Verification Error on RLVR