# KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent Evaluation arXiv: https://arxiv.org/abs/2604.08455 · HTML: https://mlsanity.com/paper/2604.08455/ ## Key facts - Cited by 21 papers in the map (data through 14 Oct 2026). - Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. - Builds on 26 of its 29 references that are in the map. - First posted on arXiv 9 Apr 2026. ## Placed in - LLM agents and tool use / [GUI and mobile agents](https://mlsanity.com/agents/gui-and-mobile-agents/) ## Builds on (most cited first, 26 of 26) - [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025) - [UI-TARS: Pioneering Automated GUI Interaction with Native Agents](https://mlsanity.com/paper/2501.12326/) (2025) - [AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents](https://mlsanity.com/paper/2405.14573/) (2024) - [On the Effects of Data Scale on UI Control Agents](https://mlsanity.com/paper/2406.03679/) (2024) - [Mobile-Agent-v3: Fundamental Agents for GUI Automation](https://mlsanity.com/paper/2508.15144/) (2025) - [UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning](https://mlsanity.com/paper/2503.21620/) (2025) - [AndroidLab: Training and Systematic Benchmarking of Android Autonomous Agents](https://mlsanity.com/paper/2410.24024/) (2024) - [UI-Venus Technical Report: Building High-performance UI Agents with RFT](https://mlsanity.com/paper/2508.10833/) (2025) - [MAI-UI Technical Report: Real-World Centric Foundation GUI Agents](https://mlsanity.com/paper/2512.22047/) (2025) - [SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation](https://mlsanity.com/paper/2410.15164/) (2024) - [Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents](https://mlsanity.com/paper/2602.16855/) (2026) - [MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments](https://mlsanity.com/paper/2512.19432/) (2025) - [GUI-G2: Gaussian Reward Modeling for GUI Grounding](https://mlsanity.com/paper/2507.15846/) (2025) - [A Survey on (M)LLM-Based GUI Agents](https://mlsanity.com/paper/2504.13865/) (2025) - [UI-Venus-1.5 Technical Report](https://mlsanity.com/paper/2602.09082/) (2026) - [Step-GUI Technical Report](https://mlsanity.com/paper/2512.15431/) (2025) - [LLM-Powered GUI Agents in Phone Automation: Surveying Progress and Prospects](https://mlsanity.com/paper/2504.19838/) (2025) - [UI-S1: Advancing GUI Automation via Semi-online Reinforcement Learning](https://mlsanity.com/paper/2509.11543/) (2025) - [FingerTip 20K: A Benchmark for Proactive and Personalized Mobile LLM Agents](https://mlsanity.com/paper/2507.21071/) (2025) - [MemGUI-Bench: Benchmarking Memory of Mobile GUI Agents in Dynamic Environments](https://mlsanity.com/paper/2602.06075/) (2026) - [PersonalAlign: Hierarchical Implicit Intent Alignment for Personalized GUI Agent with Long-Term User-Centric Records](https://mlsanity.com/paper/2601.09636/) (2026) - [Proactive Agent Research Environment: Simulating Active Users to Evaluate Proactive Assistants](https://mlsanity.com/paper/2604.00842/) (2026) - [GEM: Gaussian Embedding Modeling for Out-of-Distribution Detection in GUI Agents](https://mlsanity.com/paper/2505.12842/) (2025) - [Me-Agent: A Personalized Mobile Agent with Two-Level User Habit Learning for Enhanced Interaction](https://mlsanity.com/paper/2601.20162/) (2026) - [PIRA-Bench: A Transition from Reactive GUI Agents to GUI-based Proactive Intent Recommendation Agents](https://mlsanity.com/paper/2603.08013/) (2026) - [ProactiveMobile: A Comprehensive Benchmark for Boosting Proactive Intelligence on Mobile Devices](https://mlsanity.com/paper/2602.21858/) (2026) ## Built on by (most cited first, 21 of 21) - [VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactions](https://mlsanity.com/paper/2605.27141/) (2026) - [UI-MOPD: Multi-Platform On-Policy Distillation for Unified GUI Agents](https://mlsanity.com/paper/2607.04425/) (2026) - [π-Bench: Evaluating Proactive Personal Assistant Agents in Long-Horizon Workflows](https://mlsanity.com/paper/2605.14678/) (2026) - [Xiaomi-GUI-0 Technical Report](https://mlsanity.com/paper/2606.31410/) (2026) - [UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimization](https://mlsanity.com/paper/2604.13822/) (2026) - [Ask Now, Use Later: Benchmarking the Proactivity Gap in Long-Lived LLM Agents](https://mlsanity.com/paper/2605.28108/) (2026) - [UI-Venus-2 Technical Report](https://mlsanity.com/paper/2609.00028/) (2026) - [STAMP: Training Explicit Memory for Mobile GUI Agents in Controllable and Scalable Virtual Environments](https://mlsanity.com/paper/2605.29324/) (2026) - [AndroidReality: How Far Are Mobile Agents from the Real World?](https://mlsanity.com/paper/2608.07775/) (2026) - [Agent-G2: Gaussian Guidance for Agentic Reinforcement Learning](https://mlsanity.com/paper/2608.23318/) (2026) - [Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability](https://mlsanity.com/paper/2605.07110/) (2026) - [SocialPersona: Benchmarking Personalized Profiling and Response with Multimodal Social-Media Context](https://mlsanity.com/paper/2606.26654/) (2026) - [REDDIT: Forgetting-Resistant Correction of Timestamp Drift in ASR via Replay-Based Distribution Editing](https://mlsanity.com/paper/2607.05364/) (2026) - [The Severance Problem: LLMs are Unaware of the Person Beyond the Prompt](https://mlsanity.com/paper/2607.14250/) (2026) - [QUMem: Personalized Memory for Query-Conditioned User-State Inference in LLM Agents](https://mlsanity.com/paper/2608.16168/) (2026) - [ADeptS-Bench: Measuring the Trustworthiness of Computer Use Agents Across Devices](https://mlsanity.com/paper/2608.26204/) (2026) - [ElderBench: Benchmarking Autonomous Mobile Agents for Older Adults](https://mlsanity.com/paper/2609.04850/) (2026) - [Relevance Does Not Imply Applicability: Experience Activation for Personal GUI Agents](https://mlsanity.com/paper/2609.33304/) (2026) - [Foundations of Proactive Agents: Principles, Technical Layers, and Proactivity-Gym](https://mlsanity.com/paper/2609.37267/) (2026) - [What Did the Agent Actually Do? Evidence-Grounded Oversight for Long-Horizon Agents](https://mlsanity.com/paper/2610.06406/) (2026) - [Can Agents Work for Everyone? Cross-User Reliability for Mobile GUI Agents in Personalized User Interfaces](https://mlsanity.com/paper/2610.07972/) (2026) ## Abstract (arXiv) Personalized mobile agents that infer user preferences and calibrate proactive assistance hold great promise as everyday digital assistants, yet existing benchmarks fail to capture what this requires. Prior work evaluates preference recovery from static histories or intent prediction from fixed contexts. Neither tests whether an agent can elicit missing preferences through interaction, nor whether it can decide when to intervene, seek consent, or remain silent in a live GUI environment. We introduce KnowU-Bench, an online benchmark for personalized mobile agents built on a reproducible Android emulation environment, covering 42 general GUI tasks, 86 personalized tasks, and 64 proactive tasks. Unlike prior work that treats user preferences as static context, KnowU-Bench hides the user profile from the agent and exposes only behavioral logs, forcing genuine preference inference rather than context lookup. To support multi-turn preference elicitation, it instantiates an LLM-driven user simulator grounded in structured profiles, enabling realistic clarification dialogues and proactive consent handling. Beyond personalization, KnowU-Bench provides comprehensive evaluation of the complete proactive decision chain, including grounded GUI execution, consent negotiation, and post-rejection restraint, evaluated through a hybrid protocol combining rule-based verification with LLM-as-a-Judge scoring. Our experiments reveal a striking degradation: agents that excel at explicit task execution fall below 50% under vague instructions requiring user preference inference or intervention calibration, even for frontier models like Claude Sonnet 4.6. The core bottlenecks are not GUI navigation but preference acquisition and intervention calibration, exposing a fundamental gap between competent interface operation and trustworthy personal assistance.