# Toward a Human-Centered Evaluation Framework for Trustworthy LLM-Powered GUI Agents arXiv: https://arxiv.org/abs/2504.17934 ยท HTML: https://mlsanity.com/paper/2504.17934/ ## Key facts - Cited by 11 papers in the map (data through 14 Oct 2026). - Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. - Builds on 21 of its 56 references that are in the map. - First posted on arXiv 24 Apr 2025. ## Placed in - LLM agents and tool use / [GUI and mobile agents](https://mlsanity.com/agents/gui-and-mobile-agents/) ## Builds on (most cited first, 21 of 21) - [GPT-4V(ision) is a Generalist Web Agent, if Grounded](https://mlsanity.com/paper/2401.01614/) (2024) - [DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models](https://mlsanity.com/paper/2306.11698/) (2023) - [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020) - [AppAgent: Multimodal Agents as Smartphone Users](https://mlsanity.com/paper/2312.13771/) (2023) - [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020) - [Auto-GPT for Online Decision Making: Benchmarks and Additional Opinions](https://mlsanity.com/paper/2306.02224/) (2023) - [PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action](https://mlsanity.com/paper/2409.00138/) (2024) - [Large Language Model-Brained GUI Agents: A Survey](https://mlsanity.com/paper/2411.18279/) (2024) - [UFO: A UI-Focused Agent for Windows OS Interaction](https://mlsanity.com/paper/2402.07939/) (2024) - [Attacking Vision-Language Computer Agents via Pop-ups](https://mlsanity.com/paper/2411.02391/) (2024) - [AutoGLM: Autonomous Foundation Agents for GUIs](https://mlsanity.com/paper/2411.00820/) (2024) - ["It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents](https://mlsanity.com/paper/2309.11653/) (2023) - [Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors](https://mlsanity.com/paper/2105.02980/) (2021) - [PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models](https://mlsanity.com/paper/2311.04044/) (2023) - [LlamaTouch: A Faithful and Scalable Testbed for Mobile UI Task Automation](https://mlsanity.com/paper/2404.16054/) (2024) - [AutoDroid-V2: Boosting SLM-based GUI Agents via Code Generation](https://mlsanity.com/paper/2412.18116/) (2024) - [Exploring Autonomous Agents through the Lens of Large Language Models: A Review](https://mlsanity.com/paper/2404.04442/) (2024) - [Privacy Preserving Prompt Engineering: A Survey](https://mlsanity.com/paper/2404.06001/) (2024) - [Privacy Auditing of Large Language Models](https://mlsanity.com/paper/2503.06808/) (2025) - [An Empathy-Based Sandbox Approach to Bridge the Privacy Gap among Attitudes, Goals, Knowledge, and Behaviors](https://mlsanity.com/paper/2309.14510/) (2023) - [Privacy Leakage Overshadowed by Views of AI: A Study on Human Oversight of Privacy in Language Model Agent](https://mlsanity.com/paper/2411.01344/) (2024) ## Built on by (most cited first, 11 of 11) - [A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?](https://mlsanity.com/paper/2505.10924/) (2025) - [Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight](https://mlsanity.com/paper/2509.10723/) (2025) - [LLM Agent Meets Agentic AI: Can LLM Agents Simulate Customers to Evaluate Agentic-AI-based Shopping Assistants?](https://mlsanity.com/paper/2509.21501/) (2025) - [ReGUIDE: Data Efficient GUI Grounding via Spatial Reasoning and Search](https://mlsanity.com/paper/2505.15259/) (2025) - [Through the Lens of Human-Human Collaboration: A Configurable Research Platform for Exploring Human-Agent Collaboration](https://mlsanity.com/paper/2509.18008/) (2025) - [Overseeing Agents Without Constant Oversight: Challenges and Opportunities](https://mlsanity.com/paper/2602.16844/) (2026) - [From Human-Human Collaboration to Human-Agent Collaboration: A Vision, Design Philosophy, and an Empirical Framework for Achieving Successful Partnerships Between Humans and LLM Agents](https://mlsanity.com/paper/2602.05987/) (2026) - [Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents](https://mlsanity.com/paper/2606.13385/) (2026) - [From Prompt to Product: A Human-Centered Benchmark of Agentic App Generation Systems](https://mlsanity.com/paper/2512.18080/) (2025) - [Software Engineering for and with GUI Agent](https://mlsanity.com/paper/2608.09278/) (2026) - [Value-Sensitive Delegation in Everyday AI Agent Use: Evidence from OpenClaw](https://mlsanity.com/paper/2609.22067/) (2026) ## Abstract (arXiv) The rise of Large Language Models (LLMs) has revolutionized Graphical User Interface (GUI) automation through LLM-powered GUI agents, yet their ability to process sensitive data with limited human oversight raises significant privacy and security risks. This position paper identifies three key risks of GUI agents and examines how they differ from traditional GUI automation and general autonomous agents. Despite these risks, existing evaluations focus primarily on performance, leaving privacy and security assessments largely unexplored. We review current evaluation metrics for both GUI and general LLM agents and outline five key challenges in integrating human evaluators for GUI agent assessments. To address these gaps, we advocate for a human-centered evaluation framework that incorporates risk assessments, enhances user awareness through in-context consent, and embeds privacy and security considerations into GUI agent design and evaluation.