# CUARewardBench: A Benchmark for Evaluating Reward Models on Computer-using Agent

arXiv: https://arxiv.org/abs/2510.18596 · HTML: https://mlsanity.com/paper/2510.18596/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use.
- Builds on 24 of its 29 references that are in the map.
- First posted on arXiv 21 Oct 2025.

## Placed in

- LLM agents and tool use / [GUI and mobile agents](https://mlsanity.com/agents/gui-and-mobile-agents/)
- LLM alignment / RLHF / preference optimisation / [Process reward models for reasoning](https://mlsanity.com/alignment/process-reward-models-for-reasoning/)

## Builds on (most cited first, 24 of 24)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [UI-TARS: Pioneering Automated GUI Interaction with Native Agents](https://mlsanity.com/paper/2501.12326/) (2025)
- [RewardBench: Evaluating Reward Models for Language Modeling](https://mlsanity.com/paper/2403.13787/) (2024)
- [Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents](https://mlsanity.com/paper/2410.05243/) (2024)
- [OS-ATLAS: A Foundation Action Model for Generalist GUI Agents](https://mlsanity.com/paper/2410.23218/) (2024)
- [Seed1.5-VL Technical Report](https://mlsanity.com/paper/2505.07062/) (2025)
- [Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction](https://mlsanity.com/paper/2412.04454/) (2024)
- [UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2509.02544/) (2025)
- [Mobile-Agent-v3: Fundamental Agents for GUI Automation](https://mlsanity.com/paper/2508.15144/) (2025)
- [Agent S: An Open Agentic Framework that Uses Computers Like a Human](https://mlsanity.com/paper/2410.08164/) (2024)
- [RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Style](https://mlsanity.com/paper/2410.16184/) (2024)
- [OpenCUA: Open Foundations for Computer-Use Agents](https://mlsanity.com/paper/2508.09123/) (2025)
- [Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents](https://mlsanity.com/paper/2504.00906/) (2025)
- [Scaling Computer-Use Grounding via User Interface Decomposition and Synthesis](https://mlsanity.com/paper/2505.13227/) (2025)
- [AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories](https://mlsanity.com/paper/2504.08942/) (2025)
- [GTA1: GUI Test-time Scaling Agent](https://mlsanity.com/paper/2507.05791/) (2025)
- [Lemur: Harmonizing Natural Language and Code for Language Agents](https://mlsanity.com/paper/2310.06830/) (2023)
- [SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience](https://mlsanity.com/paper/2508.04700/) (2025)
- [Multimodal RewardBench: Holistic Evaluation of Reward Models for Vision Language Models](https://mlsanity.com/paper/2502.14191/) (2025)
- [ZeroGUI: Automating Online GUI Learning at Zero Human Cost](https://mlsanity.com/paper/2505.23762/) (2025)
- [Agent-RewardBench: Towards a Unified Benchmark for Reward Modeling across Perception, Planning, and Safety in Real-World Multimodal Agents](https://mlsanity.com/paper/2506.21252/) (2025)
- [Evolving in Tasks: Empowering the Multi-modality Large Language Model as the Computer Use Agent](https://mlsanity.com/paper/2508.04037/) (2025)

## Built on by (most cited first, 18 of 18)

- [Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading](https://mlsanity.com/paper/2607.08964/) (2026)
- [OpenMobile: Building Open Mobile Agents with Task and Trajectory Synthesis](https://mlsanity.com/paper/2604.15093/) (2026)
- [StructAgent: Harness Long-horizon Digital Agents with Unified Causal Structure](https://mlsanity.com/paper/2607.11388/) (2026)
- [OS-Oracle: A Comprehensive Framework for Cross-Platform GUI Critic Models](https://mlsanity.com/paper/2512.16295/) (2025)
- [Agentic Reward Modeling: Verifying GUI Agent via Progressive Trajectory-Grounded Interaction](https://mlsanity.com/paper/2602.00575/) (2026)
- [Video-Based Reward Modeling for Computer-Use Agents](https://mlsanity.com/paper/2603.10178/) (2026)
- [CUAAudit: Meta-Evaluation of Vision-Language Models as Auditors of Autonomous Computer-Use Agents](https://mlsanity.com/paper/2603.10577/) (2026)
- [The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation](https://mlsanity.com/paper/2607.24720/) (2026)
- [OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models](https://mlsanity.com/paper/2607.28609/) (2026)
- [Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring](https://mlsanity.com/paper/2605.00754/) (2026)
- [A History-Aware Visually Grounded Critic for Computer Use Agents](https://mlsanity.com/paper/2606.11078/) (2026)
- [Reinforcement Learning for Computer-Use Agents with Autonomous Evaluation](https://mlsanity.com/paper/2606.24515/) (2026)
- [SeekJudge: A Practical Reward Framework for Reinforcement Learning in Computer-Use Agents](https://mlsanity.com/paper/2607.23263/) (2026)
- [Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability](https://mlsanity.com/paper/2605.07110/) (2026)
- [AliyunConsoleAgent: Training Web Agents in Real-World Cloud Environments via Distillation and Reinforcement Learning](https://mlsanity.com/paper/2606.09447/) (2026)
- [SCOUT: Synergizing Reasoning and Tool-Use for Computer-Use Safety](https://mlsanity.com/paper/2609.36201/) (2026)
- [DUET: Co-Evolving Solver and Grader Agents](https://mlsanity.com/paper/2610.04087/) (2026)
- [Adaptive-Group Process Rewards for GUI Agents](https://mlsanity.com/paper/2610.17710/) (2026)

## Abstract (arXiv)

Computer-using agents (CUAs) enable task completion through natural interaction with operating systems and software interfaces. While script-based verifiers are widely adopted for evaluation, they suffer from limited scalability and inability to provide step-wise assessment. Reward models offer promising alternatives, but their effectiveness on CUA evaluation remains largely underexplored. To address this gap, we present CUARewardBench, comprising four key contributions: (1) First-ever Comprehensive CUA Reward Benchmark: We introduce the first benchmark for evaluating both outcome reward models (ORM) and process reward models (PRM) on CUA tasks, enabling systematic assessment across trajectory-level and step-level evaluation. (2) Diverse, Practical and Reliable Dataset: CUARewardBench encompasses trajectories from 10 software categories and 7 agent architectures with varying performance levels (25.9%-50.8% success rates). All trajectories are expertly annotated through carefully designed protocols, with rigorous quality control to ensure reliability and practical applicability. (3) Comprehensive Analysis and Insights: Through extensive experiments across 7 vision-language models and 3 prompt templates, we reveal critical limitations of current CUA RMs, including insufficient visual reasoning capabilities, knowledge deficiencies, and the superiority of general VLMs over specialized CUA models for reward evaluation. (4) Unanimous Prompt Ensemble (UPE): Based on the insights from our comprehensive analysis, we propose UPE, a novel ensemble method that significantly enhances reward model reliability through strict unanimous voting and strategic prompt-template configurations. UPE achieves 89.8% precision and 93.3% NPV for ORM, and 81.7% precision and 85.1% NPV for PRM, substantially outperforming single VLMs and traditional ensemble approaches.