# OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models

arXiv: https://arxiv.org/abs/2607.28609 · HTML: https://mlsanity.com/paper/2607.28609/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Process reward models for reasoning, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 28 of its 74 references that are in the map.
- First posted on arXiv 30 Jul 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Process reward models for reasoning](https://mlsanity.com/alignment/process-reward-models-for-reasoning/)

## Builds on (most cited first, 28 of 28)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [SGLang: Efficient Execution of Structured Language Model Programs](https://mlsanity.com/paper/2312.07104/) (2023)
- [UI-TARS: Pioneering Automated GUI Interaction with Native Agents](https://mlsanity.com/paper/2501.12326/) (2025)
- [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026)
- [MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark](https://mlsanity.com/paper/2402.04788/) (2024)
- [Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V](https://mlsanity.com/paper/2310.11441/) (2023)
- [WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning](https://mlsanity.com/paper/2411.02337/) (2024)
- [Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale](https://mlsanity.com/paper/2409.08264/) (2024)
- [DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning](https://mlsanity.com/paper/2406.11896/) (2024)
- [OS-Copilot: Towards Generalist Computer Agents with Self-Improvement](https://mlsanity.com/paper/2402.07456/) (2024)
- [AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories](https://mlsanity.com/paper/2504.08942/) (2025)
- [GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents](https://mlsanity.com/paper/2506.03143/) (2025)
- [InSTA: Towards Internet-Scale Training For Agents](https://mlsanity.com/paper/2502.06776/) (2025)
- [Intern-S1-Pro: Scientific Multimodal Foundation Model at Trillion Scale](https://mlsanity.com/paper/2603.25040/) (2026)
- [GUI-Shepherd: Reliable Process Reward and Verification for Long-Sequence GUI Tasks](https://mlsanity.com/paper/2509.23738/) (2025)
- [CUARewardBench: A Benchmark for Evaluating Reward Models on Computer-using Agent](https://mlsanity.com/paper/2510.18596/) (2025)
- [A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond](https://mlsanity.com/paper/2403.14734/) (2024)
- [OpenMobile: Building Open Mobile Agents with Task and Trajectory Synthesis](https://mlsanity.com/paper/2604.15093/) (2026)
- [OS-Themis: A Scalable Critic Framework for Generalist GUI Rewards](https://mlsanity.com/paper/2603.19191/) (2026)
- [OS-MAP: How Far Can Computer-Using Agents Go in Breadth and Depth?](https://mlsanity.com/paper/2507.19132/) (2025)
- [CODA: Coordinating the Cerebrum and Cerebellum for a Dual-Brain Computer Use Agent with Decoupled Reinforcement Learning](https://mlsanity.com/paper/2508.20096/) (2025)
- [OdysseyArena: Benchmarking Large Language Models For Long-Horizon, Active and Inductive Interactions](https://mlsanity.com/paper/2602.05843/) (2026)
- [Autonomous Continual Learning for Environment Adaptation of Computer-Use Agents](https://mlsanity.com/paper/2602.10356/) (2026)

## Built on by (most cited first, 2 of 2)

- [UI-Venus-2 Technical Report](https://mlsanity.com/paper/2609.00028/) (2026)
- [OpenMobile-2: Building Versatile Mobile Agents with Scalable Environments and App-Native Tools](https://mlsanity.com/paper/2610.14157/) (2026)

## Abstract (arXiv)

Computer-using agents (CUAs) are advancing rapidly across the digital world. A CUA trajectory records the agent's actions, states, and reasoning. Verifying whether it fulfilled the task instruction is central to CUA evaluation, data curation, and reinforcement learning. Neither human-written verifiers nor human annotators can provide such verification at scale, so the field increasingly turns to vision-language models (VLMs) as judges of CUA trajectories. But a fundamental question has long gone unexamined: are these VLM judges reliable enough? To study it systematically, we introduce OSReward, a realistic, high-quality benchmark that evaluates VLM judges on CUA trajectories. The trajectories come from diverse agent backbones executing human-verified instructions across platforms, and are then rigorously labeled with ground-truth verdicts through multi-stage human annotation. Building on it, we derive OSReward-Hard, a challenge set concentrating genuinely hard cases, and OSReward-Multi for fine-grained efficiency and alignment scoring. The most comprehensive evaluation of VLM judges to date finds even state-of-the-art models fall short of an ideal judge, sharing a systematic leniency bias that mislabels failed runs as successes. The few reliable enough to trust are too expensive to run at scale, while affordable open models trail far behind. To close this gap, we construct and release OS-Shepherd-100K, an open corpus of reasoning-annotated trajectory judgments for the CUA community. On it, we train OS-Shepherd (9B and 35B), open reward models that supply low-cost, stable, and reliable reward signals, matching commercial judges at 30-60x lower cost than the frontier. Extensive analyses further inform the design of reliable CUA reward at scale. Our code, benchmark, dataset, and model checkpoints are available at https://os-copilot.github.io/OSReward-Home/.