mlsanityData to 7 Oct 2026

AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following

arXiv 2511.10507 ↗First posted 13 Nov 2025cs.CLcited by 46

Not on the map. Why?

Abstract (arXiv)

Recent progress in large language models (LLMs) has led to impressive performance on a range of tasks, yet advanced instruction following (IF)-especially for complex, multi-turn, and system-prompted instructions-remains a significant challenge. Rigorous evaluation and effective training for such capabilities are hindered by the lack of high-quality, human-annotated benchmarks and reliable, interpretable reward signals. In this work, we introduce AdvancedIF (we will release this benchmark soon), a comprehensive benchmark featuring over 1,600 prompts and expert-curated rubrics that assess LLMs ability to follow complex, multi-turn, and system-level instructions. We further propose RIFL (Rubric-based Instruction-Following Learning), a novel post-training pipeline that leverages rubric generation, a finetuned rubric verifier, and reward shaping to enable effective reinforcement learning for instruction following. Extensive experiments demonstrate that RIFL substantially improves the instruction-following abilities of LLMs, achieving a 6.7% absolute gain on AdvancedIF and strong results on public benchmarks. Our ablation studies confirm the effectiveness of each component in RIFL. This work establishes rubrics as a powerful tool for both training and evaluating advanced IF in LLMs, paving the way for more capable and reliable AI systems.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Builds on 35 of its 46 references that are in the map. First posted on arXiv 13 Nov 2025.

Builds on

35 of 46 references.

References · 35

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  8. 2017Deep reinforcement learning from human preferencescited by 3,846
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  10. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  11. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  12. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
23 more
  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  3. 2020Learning to summarize from human feedbackcited by 2,562
  4. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  5. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  6. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  7. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 484
  8. 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domainscited by 328
  9. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 205
  10. 2025Generalizing Verifiable Instruction Followingcited by 178
  11. 2025MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMscited by 171
  12. 2024Benchmarking Complex Instruction-Following with Multiple Constraints Compositioncited by 147
  13. 2025Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correctioncited by 140
  14. 2024InFoBench: Evaluating Instruction Following Ability in Large Language Modelscited by 128
  15. 2025Checklists Are Better Than Reward Models For Aligning Language Modelscited by 115
  16. 2023FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Modelscited by 112
  17. 2025Reinforcement Learning with Rubric Anchorscited by 108
  18. 2024Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Followingcited by 94
  19. 2024RRM: Robust Reward Model Training Mitigates Reward Hackingcited by 75
  20. 2025Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoningcited by 56
  21. 2025Prompt Curriculum Learning for Efficient LLM Post-Trainingcited by 40
  22. 2025How Many Instructions Can LLMs Follow at Once?cited by 17
  23. 2025WildIFEval: Instruction Following in the Wildcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 44.

46 papers.

Sort:

By area:

  1. 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
  2. 2026Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challengescited by 30
  3. 2026RubricEval: A Rubric-Level Meta-Evaluation Benchmark for LLM Judges in Instruction Followingcited by 19
  4. 2026EvoLM: Self-Evolving Language Models through Co-Evolved Discriminative Rubricscited by 12
  5. 2026An Imperfect Verifier is Good Enough: Learning with Noisy Rewardscited by 11
  6. 2026EvoRubrics: Dynamic Rubrics as Rewards via Adversarial Co-Evolution for LLM Reinforcement Learningcited by 11
  7. 2026CDRRM: Contrast-Driven Rubric Generation for Reliable and Interpretable Reward Modelingcited by 10
  8. 2026Reproducing, Analyzing, and Detecting Reward Hacking in Rubric-Based Reinforcement Learningcited by 9
  9. 2026EnterpriseBench Corecraft: Training Generalizable Agents on High-Fidelity RL Environmentscited by 8
  10. 2026Rubrics to Tokens: Bridging Response-level Rubrics and Token-level Rewards in Instruction Following Taskscited by 8
  11. 2026Omni-RRM: Advancing Omni Reward Modeling via Automatic Rubric-Grounded Preference Synthesiscited by 7
  12. 2026BenchPreS: A Benchmark for Context-Aware Personalized Preference Selectivity of Persistent-Memory LLMscited by 7
34 more
  1. 2026EvoRubric: Self-Evolving Rubric-Driven RL for Open-Ended Generationcited by 7
  2. 2026Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluationcited by 6
  3. 2026SibylSense: Adaptive Rubric Learning via Memory Tuning and Adversarial Probingcited by 6
  4. 2026Bootstrapping Post-training Signals for Open-ended Tasks via Rubric-based Self-play on Pre-training Textcited by 5
  5. 2026Alternating Reinforcement Learning with Contextual Rubric Rewards: Beyond the Scalarization Strategycited by 4
  6. 2026Lens: Rethinking Training Efficiency for Foundational Text-to-Image Modelscited by 4
  7. 2026Focal Reward: Balanced Reinforcement Learning under Rubric-Based Rewardscited by 3
  8. 2026RUBRIC-ARROW: Alternating Pointwise Rubric Reward Modeling for LLM Post-training in Non-verifiable Domainscited by 3
  9. 2026ComplexConstraints and Beyond: Expert Rubrics for RLVRcited by 3
  10. 2026Quark Medical Alignment: A Holistic Multi-Dimensional Alignment and Collaborative Optimization Paradigmcited by 2
  11. 2026Aligning Large Language Models with Searcher Preferencescited by 2
  12. 2026CLR-voyance: Reinforcing Open-Ended Reasoning for Inpatient Clinical Decision Support with Outcome-Aware Rubricscited by 2
  13. 2026Training Documents Reranker with Search Rubrics for Deep Research Agentcited by 2
  14. 2026Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RLcited by 2
  15. 2026FMBench: Adaptive Large Language Model Output Formattingcited by 1
  16. 2026Experience is the Best Teacher: Motivating Effective Exploration in Reinforcement Learning for LLMscited by 1
  17. 2026QUBRIC: Co-Designing Queries and Rubrics for RL Beyond Verifiable Rewardscited by 1
  18. 2026Support Vector Rubrics: Closing the Gap Between Self-Generated and Human Rubricscited by 1
  19. 2025BRAID: Bounded Reasoning for Autonomous Inference and Decisionscited by 0
  20. 2025MUSIC: MUlti-Step Instruction Contrast for Multi-Turn Reward Modelscited by 0
  21. 2026Towards Autonomous Memory Agentscited by 0
  22. 2026Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoringcited by 0
  23. 2026Co-ReAct: Rubrics as Step-Level Collaborators for ReAct Agentscited by 0
  24. 2026Beyond Rubrics: Exploration-Guided Evaluation Skills for Reward Modelingcited by 0
  25. 2026LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RLcited by 0
  26. 2026CoRT: Counterfactual Replay for Token-Level Rubric-Guided Policy Optimizationcited by 0
  27. 2026PRISM: Priority-aware Rubric Internalization via Structured Multimodal Data Synthesiscited by 0
  28. 2026DAREBench: Deployment-Aware and Reliable Evaluation of Models as Agentscited by 0
  29. 2026RLVR2: Reinforcement Learning with Verifiable Rubric-based Rankingcited by 0
  30. 2026Rufus-Air: An Open LLM Post-Training Recipecited by 0
  31. 2026A Cheap Verifier is Good Enough: LLM Post-training is Robust to Erroneous Rewardscited by 0
  32. 2026Learning to Learn from Context: Synthetic Training from Perturbed Public Documentscited by 0
  33. 2026Cross-Benchmark Transfer from RL on Agentic Coding Taskscited by 0
  34. 2026RewardWeaver: Long-Horizon Interactive Learning for Language Agents via Self-Evolving Reward Adaptationcited by 0