# CAVEAT: Towards Robust Computer-Use Agents in Incentive-Misaligned Environments

arXiv: https://arxiv.org/abs/2609.27273 · HTML: https://mlsanity.com/paper/2609.27273/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 23 Sep 2026.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 20 of 20)

- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents](https://mlsanity.com/paper/2403.02691/) (2024)
- [WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models](https://mlsanity.com/paper/2401.13919/) (2024)
- [WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?](https://mlsanity.com/paper/2403.07718/) (2024)
- [Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale](https://mlsanity.com/paper/2504.14225/) (2025)
- [WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks](https://mlsanity.com/paper/2504.18575/) (2025)
- [Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs](https://mlsanity.com/paper/2502.09597/) (2025)
- [ShoppingBench: A Real-World Intent-Grounded Shopping Benchmark for LLM-based Agents](https://mlsanity.com/paper/2508.04266/) (2025)
- [How Bad is Top-K Recommendation under Competing Content Creators?](https://mlsanity.com/paper/2302.01971/) (2023)
- [Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs](https://mlsanity.com/paper/2505.11556/) (2025)
- [Magentic Marketplace: An Open-Source Environment for Studying Agentic Markets](https://mlsanity.com/paper/2510.25779/) (2025)
- [DECEPTICON: How Dark Patterns Manipulate Web Agents](https://mlsanity.com/paper/2512.22894/) (2025)
- [A Framework for Studying AI Agent Behavior: Evidence from Consumer Choice Experiments](https://mlsanity.com/paper/2509.25609/) (2025)
- [SusBench: An Online Benchmark for Evaluating Dark Pattern Susceptibility of Computer-Use Agents](https://mlsanity.com/paper/2510.11035/) (2025)
- [Can Probabilistic Feedback Drive User Impacts in Online Platforms?](https://mlsanity.com/paper/2401.05304/) (2024)
- [Product Ranking for Revenue Maximization with Multiple Purchases](https://mlsanity.com/paper/2210.08268/) (2022)
- [Benchmarking Web Agent Safety under E-commerce Deceptive Interfaces](https://mlsanity.com/paper/2606.13686/) (2026)

## Abstract (arXiv)

Computer-use agents (CUAs) increasingly act on behalf of users online. What happens when the environments they operate in have incentives of their own? Online marketplaces, for example, may favor some products over others, steering agents away from the user's objective. Existing CUA benchmarks cover cooperative settings or explicit attacks, but do not test whether agents preserve user objectives when the environment itself has a stake in the outcome. We introduce CAVEAT, a controlled benchmark spanning nine marketplace environments and a taxonomy of eight common steering mechanisms. Across five model families, agents purchase the user-optimal product in 78.6% of matched-control episodes but only 17.3% when steering mechanisms are enabled. Larger models and more reasoning improve robustness, but substantial failures persist. Trajectory analysis and targeted ablations identify three weaknesses in how agents decide: they (1) prematurely narrow the set of alternatives they consider, (2) impose priorities the user never stated, and (3) commit before resolving decision-relevant evidence. Guided by this diagnosis, we develop CAVEAT-Harness, which targets these failures and raises the optimal purchase rate by up to 80.0 percentage points, and show that targeted post-training further improves a smaller open model. These results establish incentive robustness as a distinct challenge for delegated agents, diagnose failure modes, and show how targeted interventions can substantially improve robustness.