# AdaRubric: Task-Adaptive Rubrics for Reliable LLM Agent Evaluation and Reward Learning

arXiv: https://arxiv.org/abs/2603.21362 · HTML: https://mlsanity.com/paper/2603.21362/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 22 Mar 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)
- LLM alignment / RLHF / preference optimisation / [Process reward models for reasoning](https://mlsanity.com/alignment/process-reward-models-for-reasoning/)

## Builds on (most cited first, 20 of 20)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [Self-Rewarding Language Models](https://mlsanity.com/paper/2401.10020/) (2024)
- [Prometheus: Inducing Fine-grained Evaluation Capability in Language Models](https://mlsanity.com/paper/2310.08491/) (2023)
- [How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources](https://mlsanity.com/paper/2306.04751/) (2023)
- [JudgeLM: Fine-tuned Large Language Models are Scalable Judges](https://mlsanity.com/paper/2310.17631/) (2023)
- [FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets](https://mlsanity.com/paper/2307.10928/) (2023)
- [Autonomous Evaluation and Refinement of Digital Agents](https://mlsanity.com/paper/2404.06474/) (2024)
- [Toward Human-Like Evaluation for Natural Language Generation with Error Analysis](https://mlsanity.com/paper/2212.10179/) (2022)
- [Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments](https://mlsanity.com/paper/2505.17616/) (2025)

## Built on by (most cited first, 13 of 13)

- [SkillCoach: Self-Evolving Rubrics for Evaluating and Enhancing Agentic Skill-Use](https://mlsanity.com/paper/2607.01874/) (2026)
- [ScienceBuddy: Recursive-in-Recursive Self-Improvement for Interactive Scientific Agents](https://mlsanity.com/paper/2609.17523/) (2026)
- [From Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscape](https://mlsanity.com/paper/2606.08625/) (2026)
- [Beyond Idealized Patients: Evaluating LLMs under Challenging Patient Behaviors in Medical Consultations](https://mlsanity.com/paper/2603.29373/) (2026)
- [The Hitchhikers Guide to Rubric Quality Understanding and Enrichment](https://mlsanity.com/paper/2604.01375/) (2026)
- [When Gradients Collide: Failure Modes of Multi-Objective Prompt Optimization for LLM Judges](https://mlsanity.com/paper/2605.26046/) (2026)
- [ARBOR: Online Process Rewards via a Reusable Rubric Buffer for Search Agents](https://mlsanity.com/paper/2606.03239/) (2026)
- [Grounded Scaling: Why Agentic AI Needs Deterministic Environments](https://mlsanity.com/paper/2606.22495/) (2026)
- [Share the Judge, Learn the Deferral: Where Specialization Helps LLM Evaluation](https://mlsanity.com/paper/2607.27984/) (2026)
- [APTER: Adaptive Post-Training with Expert-Grounded Rubrics](https://mlsanity.com/paper/2608.14212/) (2026)
- [Self-Designed Evaluators and Warm Memory for Long-Horizon Agents](https://mlsanity.com/paper/2609.33717/) (2026)
- [EvoCowork: A Benchmark for Real-World Office Tasks](https://mlsanity.com/paper/2610.14828/) (2026)
- [Not All Criteria Matter: Importance-Aware Criteria Weighting for Rubric-Based LLM Evaluation](https://mlsanity.com/paper/2610.19915/) (2026)

## Abstract (arXiv)

Evaluating LLM agent trajectories is fundamentally task-specific: a code-debugging agent should be judged on Correctness and Error Handling, not on Fluency or Safety. Yet the dominant paradigm -- LLM-as-Judge with a fixed rubric -- applies the same static dimensions regardless of task, producing systematic mis-evaluation. We present AdaRubric, a framework that (i) adaptively generates task-specific evaluation rubrics from task descriptions via LLM, (ii) evaluates agent trajectories step-by-step with confidence-weighted, per-dimension scoring, and (iii) produces dense reward signals for preference learning. Three composable filtering strategies, including the novel DimensionAwareFilter that provably prevents dimension-level quality masking, yield high-quality DPO preference pairs. On WebArena, ToolBench, and AgentBench, AdaRubric achieves Pearson r = 0.79 human correlation (+0.15 over the strongest baseline), with strong reliability (Krippendorff's alpha = 0.83). DPO models trained on AdaRubric-generated pairs improve task success by +6.8-8.5% over the best baseline. AdaRubric also generalises zero-shot to unseen domains (SWE-bench) and extends to multimodal agents (VisualWebArena, OSWorld) without modification. Our code is available at: github.com/alphadl/AdaRubrics