mlsanityData to 7 Oct 2026

UXAgent: An LLM Agent-Based Usability Testing Framework for Web Design

arXiv 2502.12561 ↗First posted 18 Feb 2025cs.HC cs.CLcited by 39

On the map: LLM agents and tool use → GUI and mobile agents

Abstract (arXiv)

Usability testing is a fundamental yet challenging (e.g., inflexible to iterate the study design flaws and hard to recruit study participants) research method for user experience (UX) researchers to evaluate a web design. Recent advances in Large Language Model-simulated Agent (LLM-Agent) research inspired us to design UXAgent to support UX researchers in evaluating and reiterating their usability testing study design before they conduct the real human subject study. Our system features an LLM-Agent module and a universal browser connector module so that UX researchers can automatically generate thousands of simulated users to test the target website. The results are shown in qualitative (e.g., interviewing how an agent thinks ), quantitative (e.g., # of actions), and video recording formats for UX researchers to analyze. Through a heuristic user evaluation with five UX researchers, participants praised the innovation of our system but also expressed concerns about the future of LLM Agent-assisted UX study.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 17 of its 40 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

17 of 40 references.

References · 17

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  2. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  3. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  4. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  5. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  6. 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individualscited by 235
  7. 2024AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environmentscited by 215
  8. 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 161
  9. 2024AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agentscited by 110
  10. 2019Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AIcited by 79
  11. 2023LASER: LLM Agent with State-Space Exploration for Web Navigationcited by 66
  12. 2023AXNav: Replaying Accessibility Tests from Natural Languagecited by 31
5 more
  1. 2023An Empathy-Based Sandbox Approach to Bridge the Privacy Gap among Attitudes, Goals, Knowledge, and Behaviorscited by 18
  2. 2024Privacy Leakage Overshadowed by Views of AI: A Study on Human Oversight of Privacy in Language Model Agentcited by 14
  3. 2023Collaboration with Conversational AI Assistants for UX Evaluation: Questions and How to Ask them (Voice vs. Text)cited by 6
  4. 2024Can Large Language Models Replace Human Subjects? A Large-Scale Replication of Scenario-Based Experiments in Psychology and Managementcited by 5
  5. 2022"Merging Results Is No Easy Task": An International Survey Study of Collaborative Data Analysis Practices Among UX Practitionerscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 17; 2026: 22.

39 papers.

Sort:

By area:

  1. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 146
  2. 2025WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratchcited by 66
  3. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  4. 2025Can LLM Agents Simulate Multi-Turn Human Behavior? Evidence from Real Online Customer Behavior Datacited by 27
  5. 2025OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulationcited by 26
  6. 2025AgentA/B: Automated and Scalable Web A/BTesting with Interactive LLM Agentscited by 19
  7. 2025Shop-R1: Rewarding LLMs to Simulate Human Behavior in Online Shopping via Reinforcement Learningcited by 19
  8. 2025Customer-R1: Personalized Simulation of Human Behaviors via RL-based LLM Agent in Online Shoppingcited by 16
  9. 2025LLM Agent Meets Agentic AI: Can LLM Agents Simulate Customers to Evaluate Agentic-AI-based Shopping Assistants?cited by 15
  10. 2026GameGen-Verifier: Parallel Keypoint-Based Verification for LLM-Generated Games via Runtime State Injectioncited by 9
  11. 2025AI Agents for Web Testing: A Case Study in the Wildcited by 5
  12. 2025See, Think, Act: Online Shopper Behavior Simulation with VLM Agentscited by 5
27 more
  1. 2025Through the Lens of Human-Human Collaboration: A Configurable Research Platform for Exploring Human-Agent Collaborationcited by 4
  2. 2026WebTestPilot: Agentic End-to-End Web Testing against Natural Language Specification by Inferring Oracles with Symbolized GUI Elementscited by 4
  3. 2026WebTestBench: Evaluating Computer-Use Agents towards End-to-End Automated Web Testingcited by 4
  4. 2026Agents at Risk: How Users Unwittingly Undermine LLM Safetycited by 3
  5. 2026The Behavioral Fabric of LLM-Powered GUI Agents: Human Values and Interaction Outcomescited by 3
  6. 2026I-WebGenBench : Evaluating Interactivity in LLM-Generated Scientific Web Applicationscited by 3
  7. 2026ProEval: Proactive Failure Discovery and Efficient Performance Estimation for Generative AI Evaluationcited by 2
  8. 2026Training Computer Use Agents to Assess the Usability of Graphical User Interfacescited by 2
  9. 2026DiagEval: Trajectory-Conditioned Diagnosis for Reliable Software Evaluation with GUI Agentscited by 2
  10. 2025A Blueprint for AI-Driven Software Quality: Integrating LLMs with Established Standardscited by 1
  11. 2025Free Lunch for User Experience: Crowdsourcing Agents for Scalable User Studiescited by 1
  12. 2025Synthetic Cognitive Walkthrough: Aligning Large Language Model Performance with Human Cognitive Walkthroughcited by 1
  13. 2026StreetDesignAI: Broadening Designer Perspectives Through Multi-Persona Evaluation of Cycling Infrastructurecited by 1
  14. 2026Interview-Informed Generative Agents for Product Discovery: A Validation Studycited by 1
  15. 2026SimPersona: Learning Discrete Buyer Personas from Raw Clickstreams for Grounded E-Commerce Agentscited by 1
  16. 2026SimGym: A Framework for A/B Test Simulation in E-Commerce with Traffic-Grounded VLM Agentscited by 1
  17. 2026PerceptUI: LLM Agents as Human-Aligned Synthetic Users for UI/UX Evaluationcited by 1
  18. 2026VISTA: A Versatile Interactive User Simulation Toolkit for Agent Evaluationcited by 1
  19. 2025OptAgent: Optimizing Query Rewriting for E-commerce via Multi-Agent Simulationcited by 0
  20. 2025Simulating Teams with LLM Agents: Interactive 2D Environments for Studying Human-AI Dynamicscited by 0
  21. 2026Textile IR: A Bidirectional Intermediate Representation for Physics-Aware Fashion CADcited by 0
  22. 2026After Talking with 1,000 Personas: Learning Preference-Aligned Proactive Assistants From Large-Scale Persona Interactionscited by 0
  23. 2026Same Outcomes, Different Journeys: A Trace-Level Framework for Comparing Human and GUI-Agent Behavior in Production Search Systemscited by 0
  24. 2026CHORUS: An Agentic Framework for Generating Realistic Deliberation Datacited by 0
  25. 2026Architecture-Sensitive Supervised Fine-Tuning for Screen-Conditioned Action Prediction: A PiSAR Benchmarkcited by 0
  26. 2026The Judge Knows When It Knows: Calibrated Abstention for LLM-Based A/B-Test Predictioncited by 0
  27. 2026SimTrace: Grounded Multimodal User Trajectories Generation for Online User Modelingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agents
  2. 2025Understanding User Experiences of Computer Use Agents: Design Space and Opportunities for Building Agent UX Prototypes
  3. 2026UXCascade: Scalable Usability Testing with Simulated User Agents
  4. 2026Mapping the Design Space of User Experience for Computer Use Agents
  5. 2026Automatic multimodal UX improvement recommendations from LLM agent user simulations
  6. 2026From SERPs to Agents: A Platform for Comparative Studies of Information Interaction
  7. 2026HxAgent: Iterative Agent Planning for End-to-End Web Application Testing
  8. 2025Synthetic Heuristic Evaluation: A Comparison between AI- and Human-Powered Usability Evaluation