Data to 7 Oct 2026

AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories

arXiv 2504.08942 ↗First posted 11 Apr 2025cs.LG cs.AI cs.CLcited by 99

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Web agents enable users to perform tasks on web browsers through natural language interaction. Evaluating web agents trajectories is an important problem, since it helps us determine whether the agent successfully completed the tasks. Rule-based methods are widely used for this purpose, but they are challenging to extend to new tasks and may not always recognize successful trajectories. We may achieve higher accuracy through human evaluation, but the process would be substantially slower and more expensive. Automatic evaluations with LLMs may avoid the challenges of designing new rules and manually annotating trajectories, enabling faster and cost-effective evaluation. However, it is unclear how effective they are at evaluating web agents. To this end, we propose AgentRewardBench, the first benchmark to assess the effectiveness of LLM judges for evaluating web agents. AgentRewardBench contains 1302 trajectories across 5 benchmarks and 4 LLMs. Each trajectory in AgentRewardBench is reviewed by an expert, who answers questions pertaining to the success, side effects, and repetitiveness of the agent. Using our benchmark, we evaluate 12 LLM judges and find that no single LLM excels across all benchmarks. We also find that the rule-based evaluation used by common benchmarks tends to underreport the success rate of web agents, highlighting a key weakness of rule-based evaluation and the need to develop more flexible automatic evaluations. We release the benchmark at: https://agent-reward-bench.github.io

Key facts. Cited by 99 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 25 of its 38 references that are in the map. First posted on arXiv 11 Apr 2025.

Builds on

25 of 38 references.

Closest ancestors · 10

Same direction, newest first

Other references · 15

2023GPT-4 Technical Reportcited by 18,132
2024The Llama 3 Herd of Modelscited by 14,192
2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
2024GPT-4o System Cardcited by 4,467
2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 282
2024WebLINX: Real-World Website Navigation with Multi-Turn Dialoguecited by 177
2024WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Taskscited by 81
2023From Pixels to UI Actions: Learning to Follow Instructions via Graphical User Interfacescited by 67

3 more

2018Learning to Navigate the Webcited by 60
2024BAGEL: Bootstrapping Agents by Guiding Exploration with Languagecited by 34
2025PairBench: Are Vision-Language Models Reliable at Comparing What They See?cited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 20; 2026: 79.

99 papers.

Sort:

By area:

2025Agentic Reinforced Policy Optimizationcited by 166
2025RewardBench 2: Advancing Reward Model Evaluationcited by 136
2025OS-Harm: A Benchmark for Measuring Safety of Computer Use Agentscited by 104
2025SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experiencecited by 68
2025AgentSynth: Scalable Task Generation for Generalist Computer-Use Agentscited by 41
2025Agentic Entropy-Balanced Policy Optimizationcited by 39
2026AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agentscited by 28
2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
2026AutoWebWorld: Synthesizing Infinite Verifiable Web Environments via Finite State Machinescited by 20
2026Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verificationcited by 19
2025CUARewardBench: A Benchmark for Evaluating Reward Models on Computer-using Agentcited by 16
2025WAREX: Web Agent Reliability Evaluation on Existing Benchmarkscited by 14

38 more

2025E-valuator: Reliable Agent Verifiers with Sequential Hypothesis Testingcited by 14
2026A Survey on LLM-based Conversational User Simulationcited by 14
2025MEMTRACK: Evaluating Long-Term Memory and State Tracking in Multi-Platform Dynamic Agent Environmentscited by 13
2026ToolPRMBench: Evaluating and Advancing Process Reward Models for Tool-using Agentscited by 13
2026Exploring Reasoning Reward Model for Agentscited by 13
2026The Art of Building Verifiers for Computer Use Agentscited by 11
2026Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?cited by 10
2025Reinforcement Learning for Large Model: A Surveycited by 8
2026IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluationcited by 8
2025What Is Your Agent's GPA? A Framework for Evaluating Agent Goal-Plan-Action Alignmentcited by 7
2026WebArbiter: A Principle-Guided Reasoning Process Reward Model for Web Agentscited by 7
2026Aligning Agents via Planning: A Benchmark for Trajectory-Level Reward Modelingcited by 7
2025CODA: Coordinating the Cerebrum and Cerebellum for a Dual-Brain Computer Use Agent with Decoupled Reinforcement Learningcited by 6
2026Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluationcited by 6
2026Generalization in Online Reinforcement Learning for Mobile Agentscited by 6
2026GUIDE: Interpretable GUI Agent Evaluation via Hierarchical Diagnosiscited by 6
2026OpenWebRL: Demystifying Online Multi-turn Reinforcement Learning for Visual Web Agentscited by 6
2026BigFinanceBench: A Workflow-Grounded Benchmark for Financial-Research Agentscited by 6
2026Fara-1.5: Scalable Learning Environments for Computer Use Agentscited by 5
2025Libra: Assessing and Improving Reward Model by Learning to Thinkcited by 4
2025RoleRMBench & RoleRM: Towards Reward Modeling for Profile-Based Role Play in Dialogue Systemscited by 4
2026Autonomous Continual Learning for Environment Adaptation of Computer-Use Agentscited by 4
2026Can Agent Benchmarks Support Their Scores? Evidence-Supported Bounds for Interactive-Agent Evaluationcited by 4
2026PEEK: Context Map as an Orientation Cache for Long-Context LLM Agentscited by 4
2026Catching One in Five: LLM-as-Judge Blind Spots in Production Multi-Turn Transaction Agentscited by 4
2026GroundEval: A Deterministic Replacement for LLM-as-Judge in Stateful Agent Evaluationcited by 4
2026GRADE: Graph Representation of LLM Agent Dependency and Executioncited by 4
2026StructAgent: Harness Long-horizon Digital Agents with Unified Causal Structurecited by 4
2026Video-Based Reward Modeling for Computer-Use Agentscited by 3
2026Structured Distillation of Web Agent Capabilities Enables Generalizationcited by 3
2026Where Did It Go Wrong? Process-Level Evaluation of Web Agents with Semantic State Trackingcited by 3
2026How Benchmarks Mis-Score Computer-Use Agentscited by 3
2025NaturalGAIA: A Verifiable Benchmark and Hierarchical Framework for Long-Horizon GUI Taskscited by 2
2026CUBE: A Standard for Unifying Agent Benchmarkscited by 2
2026Agentic CLEAR: Automating Multi-Level Evaluation of LLM Agentscited by 2
2026Skill-Guided Continuation Distillation for GUI Agentscited by 2
2026SLBench: Evaluating How LLM Agents Follow Logical Relations in Skillscited by 2
2026Open Evaluation Agent: Efficient and Promptable Evaluation of Visual Generative Modelscited by 2

50 most cited of 99 All 99 (JSON)

Similar, not cited

Close in wording, no citation either way