mlsanityData to 14 Oct 2026

ComplexConstraints and Beyond: Expert Rubrics for RLVR

arXiv 2606.09118 ↗First posted 8 Jun 2026cs.AIcited by 3

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Evaluation protocols can lag behind LLM capabilities. Programmatically verified benchmarks cover narrow surface constraints, whereas real-world instruction following and agentic workflows require judging semantic, contextual, and policy-dependent behavior. We study expert-curated rubric-based evaluation as a unified mechanism for measurement and reinforcement-learning rewards across two settings: complex instruction following and enterprise agentic tasks. We identify rubric-design choices that affect reward quality, including maximum viable atomicity, intent-aware criterion design, and LLM-judge calibration. We introduce ComplexConstraints, an expert-curated instruction-following suite comprising a public 75-prompt benchmark with 1,559 rubric criteria and a disjoint 1,000-prompt training set, with 10-40 atomic criteria per prompt. Empirically, rubric rewards improve training in both fixed task datasets, such as ComplexConstraints, and stateful RL environments, such as CoreCraft. Training a 4B model on ComplexConstraints improves mean criterion pass rate by +15.5 pp on a held-out split, bringing it within 0.5 pp of the untrained baseline of a roughly 60x larger Qwen3 model, and the gains transfer to external benchmarks the model never saw during training: +8.4 pp on AdvancedIF and +10.1 pp on MultiChallenge. In CoreCraft, rubric-reward RL likewise transfers to out-of-distribution benchmarks (+4.5 pp BFCL, +7.4 pp tau^2-Bench, +6.8 pp Toolathlon). These results show that expert-authored rubrics provide effective evaluation targets and scalable reward signals for improving LLM instruction following and agentic behavior.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 30 of its 33 references that are in the map. First posted on arXiv 8 Jun 2026.

Builds on

30 of 33 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025ToolRL: Reward is All Tool Learning Needs16 Apr

Other references · 29

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  3. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  4. 2024A Survey on LLM-as-a-Judgecited by 1,423
  5. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  6. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  7. 2023LIMA: Less Is More for Alignmentcited by 1,100
  8. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
  9. 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
  10. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 429
  11. 2025Generalizing Verifiable Instruction Followingcited by 194
  12. 2025MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMscited by 180
17 more
  1. 2024Benchmarking Complex Instruction-Following with Multiple Constraints Compositioncited by 152
  2. 2024InFoBench: Evaluating Instruction Following Ability in Large Language Modelscited by 135
  3. 2025Checklists Are Better Than Reward Models For Aligning Language Modelscited by 128
  4. 2023FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Modelscited by 117
  5. 2025The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Executioncited by 85
  6. 2025Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluationcited by 60
  7. 2024Can You Trust LLM Judgments? Reliability of LLM-as-a-Judgecited by 56
  8. 2025AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Followingcited by 51
  9. 2025VerIF: Verification Engineering for Reinforcement Learning in Instruction Followingcited by 44
  10. 2025Neither Valid nor Reliable? Investigating the Use of LLMs as Judgescited by 39
  11. 2025Measuring Agents in Productioncited by 37
  12. 2026RubricBench: Aligning Model-Generated Rubrics with Human Standardscited by 28
  13. 2026Autorubric: A Unifying Framework for Rubric-Based LLM Evaluation on Non-Verifiable Taskscited by 26
  14. 2025An Empirical Study of LLM-as-a-Judge: How Design Choices Impact Evaluation Reliabilitycited by 19
  15. 2026RubricRAG: Towards Interpretable and Reliable LLM Evaluation via Domain Knowledge Retrieval for Rubric Generationcited by 11
  16. 2026EnterpriseBench Corecraft: Training Generalizable Agents on High-Fidelity RL Environmentscited by 8
  17. 2026The Hierarchy of Agentic Capabilities: Evaluating Frontier Models on Realistic RL Environmentscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026ATLAS: Dual-Horizon Diagnostic Evaluation for Industrial Tool-Use Agentscited by 0
  2. 2026Cross-Benchmark Transfer from RL on Agentic Coding Taskscited by 0
  3. 2026DAYJOB: A Benchmark for Long-Horizon Professional Workcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Reinforcement Learning with Rubric Anchors
  2. 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains
  3. 2026Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVR
  4. 2026RLVR2: Reinforcement Learning with Verifiable Rubric-based Ranking
  5. 2026From Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscape
  6. 2026When Rubrics Fail: Error Enumeration as Reward in Reference-Free RL Post-Training for Virtual Try-On
  7. 2026Think-with-Rubrics: From External Evaluator to Internal Reasoning Guidance
  8. 2026Step-wise Rubric Rewards for LLM Reasoning