mlsanityData to 7 Oct 2026

GUI and mobile agents

504 papers2 this weekHigh confidence in this grouping: the papers form one clear topicHigh confidence in this grouping: the papers form one clear topic · About

Keywords: gui, gui agents, agents, mobile, agent

Key facts. GUI and mobile agents: 504 papers on arXiv since 2023 (data through 7 Oct 2026). 210 of its 504 mapped papers (42%) cite “UI-TARS: Pioneering Automated GUI Interaction with Native Agents” (2025, arXiv 2501.12326). 201 of its 504 mapped papers (40%) cite “CogAgent: A Visual Language Model for GUI Agents” (2023, arXiv 2312.08914). 34 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

34 new papers, 17 Sep – 7 Oct; 2 this week.

  1. This week · 2
  2. Can Agents Work for Everyone? Cross-User Reliability for Mobile GUI Agents in Personalized User Interfaces
  3. Imagine to Act: High-Fidelity Data Synthesis via Image Editing World Model for Scalable GUI Agent Training
  4. Earlier · 32
  5. Recurrent Latent Visual Search for GUI Grounding
  6. CUAWright: A Minimal Unified Interface for Digital Agents
  7. WebUIProof: Benchmarking WebUI Code Generators with UI-Agent Execution Harness
  8. DeskForge: Dense Supervision from Desktop Environments for Computer-Use Agents
  9. Not All Experience Belongs in the Weights: Component Routing for Self-Improving GUI Agents
  10. AutoGUIWorld: Image Generators as Visual World Models for GUI Agent
  11. GUI-HARVEST: Self-Improving GUI Agents through Evidence-Driven Harness Evolution
  12. cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agents
All 34
  1. Learning Reliable GUI Agents under Imperfect Priors
  2. HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use Agents
  3. Absorbed in Inertia: Activation Analysis for Computer-Use Agents
  4. MatToolBench: Benchmarking Multimodal Agents in Real-World Materials Science Workflows also fits: Multimodal and embodied agents
  5. SCA: Spatial Credit Assignment for Reinforcement Learning of GUI Agents
  6. PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience Retrieval
  7. GUITAR: Structured Failure Diagnosis of GUI Agents via State Transitions
  8. Structured Interaction, Visual Localization, and Robust Execution for Complex Web Tasks: A Technical Report on the WebRetriever Challenge also fits: Multimodal and embodied agents
  9. PhoneCLI: From App Interfaces to Callable Commands for Mobile Agents
  10. WebPageBench: Event-Level Verification and Controlled UI-Variant Generation for Web Agents
  11. LongPuzzleBench: Evaluating GUI Agents on Long-Horizon Visual Puzzles
  12. One Rollout Is All You Get: Fully Test-Time Adaptation for GUI Agents
  13. Same Tasks, Different Apps: Why Mobile GUI Agents Fail to Generalize?
  14. Relevance Does Not Imply Applicability: Experience Activation for Personal GUI Agents
  15. DashAct: A Progressive Diagnostic Benchmark for GUI Agents in Interactive Dashboard Analysis
  16. PastForward: Faster On-Device GUI Agents via Computational Experience Reuse
  17. From Tapping to Hopping: Augmenting Mobile GUI Agents with App-Native Deeplinks
  18. Jev-Mobile: Jev as an Executor for Mobile GUI Agents
  19. Learn How to Act from Your Own Interactions: On-Policy Self-Distillation for GUI Agents
  20. Towards Omni-dimensional GUI Agent Navigation with Masked Trajectory Prediction
  21. OSWorld-Pro: Process-based Evaluation for Computer Use Agents
  22. How Many Pixels Is a Digit Worth? Place-Aware Coordinate Entropy for GUI Agent Confidence Estimation
  23. MintAct: A Unified Visual Agent for Digital Environments also fits: Multimodal and embodied agents
  24. RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents

Foundations

Most cited by papers here

  1. UI-TARS: Pioneering Automated GUI Interaction with Native Agents2025cited by 210 here
  2. CogAgent: A Visual Language Model for GUI Agents2023cited by 201 here
  3. SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents2024cited by 195 here
  4. OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments2024cited by 179 here
  5. AppAgent: Multimodal Agents as Smartphone Users2023cited by 156 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 68 here
  2. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 188 here
  3. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 148 here
  4. 2023You Only Look at Screens: Multimodal Chain-of-Action Agentscited by 72 here
  5. 2023CogAgent: A Visual Language Model for GUI Agentscited by 201 here
  6. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 126 here
  7. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 195 here
  8. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 179 here
  9. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 149 here
  10. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agentscited by 154 here
  11. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 210 here
  12. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 89 here
12 more it builds on, not in this chain
  1. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 156 here
  2. 2024VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Taskscited by 89 here
  3. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 114 here
  4. 2024Android in the Zoo: Chain-of-Action-Thought for GUI Agentscited by 85 here
  5. 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 82 here
  6. 2024On the Effects of Data Scale on UI Control Agentscited by 101 here
  7. 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learningcited by 71 here
  8. 2024AMEX: Android Multi-annotation Expo Dataset for Mobile GUI Agentscited by 66 here
  9. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 82 here
  10. 2025Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agentscited by 67 here
  11. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 90 here
  12. 2025Mobile-Agent-v3: Fundamental Agents for GUI Automationcited by 84 here

The 24 most cited of 1,083.

Browse

30 of 504, ranked 6–35

  1. 6OS-ATLAS: A Foundation Action Model for Generalist GUI Agents2024154
  2. 7Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents2024149
  3. 8Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception2024114
  4. 9On the Effects of Data Scale on UI Control Agents2024101
  5. 10ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use202590
  6. 11GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents202589
  7. 12Android in the Zoo: Chain-of-Action-Thought for GUI Agents202485
  8. 13Mobile-Agent-v3: Fundamental Agents for GUI Automation202584
  9. 14Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration202482
  10. 15Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale202482
All 30
  1. 16You Only Look at Screens: Multimodal Chain-of-Action Agents202372
  2. 17DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning202471
  3. 18Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents202567
  4. 19AMEX: Android Multi-annotation Expo Dataset for Mobile GUI Agents202466
  5. 20UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning202564
  6. 21UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning202564
  7. 22Large Language Model-Brained GUI Agents: A Survey202460
  8. 23OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Web202459
  9. 24AndroidLab: Training and Systematic Benchmarking of Android Autonomous Agents202459
  10. 25OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis202458
  11. 26InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners202556
  12. 27Agent S: An Open Agentic Framework that Uses Computers Like a Human202454
  13. 28AutoGLM: Autonomous Foundation Agents for GUIs202454
  14. 29GUI Agents with Foundation Models: A Comprehensive Survey202453
  15. 30Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks202550
  16. 31MAI-UI Technical Report: Real-World Centric Foundation GUI Agents202550
  17. 32Scaling Computer-Use Grounding via User Interface Decomposition and Synthesis202548
  18. 33OS-Copilot: Towards Generalist Computer Agents with Self-Improvement202448
  19. 34SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation202448
  20. 35GUICourse: From General Vision Language Models to Versatile GUI Agents202446