mlsanityData to 14 Oct 2026

GTA1: GUI Test-time Scaling Agent

arXiv 2507.05791 ↗First posted 8 Jul 2025cs.AIcited by 106

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Graphical user interface (GUI) agents autonomously complete tasks across platforms (eg, Linux) by sequentially decomposing user instructions into action proposals that iteratively interact with visual elements in the evolving environment. However, two main challenges arise: i) planning (ie, the action proposal sequence) under expansive action space, where selecting an appropriate plan is non-trivial, as many valid ones may exist; ii) accurately grounding actions in complex and high-resolution interfaces, ie, precisely interacting with visual targets. This paper investigates the aforementioned challenges with our GUI Test-time Scaling Agent, namely GTA1. First, we conduct test-time scaling to select the most appropriate action proposal: at each step, multiple candidate proposals are sampled and evaluated and selected by a judge model. It trades off computation for better decision quality by concurrent sampling. Second, we propose a model that improves grounding of the selected action proposals to its corresponding visual elements. Our key insight is that reinforcement learning (RL) facilitates grounding through inherent objective alignments, rewarding successful clicks on interface elements. Experimentally, GTA1 achieves state-of-the-art performance on both grounding and agent task execution benchmarks. The code and models are released here.

Key facts. Cited by 106 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 23 of its 40 references that are in the map. First posted on arXiv 8 Jul 2025.

Builds on

23 of 40 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents21 May
  2. 2025InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners19 Apr
  3. 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning27 Mar
  4. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction5 Dec

Other references · 19

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
  6. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
  7. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
  8. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
  9. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 382
  10. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agentscited by 378
  11. 2025Seed1.5-VL Technical Reportcited by 373
  12. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 254
7 more
  1. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 244
  2. 2024OmniParser for Pure Vision Based GUI Agentcited by 198
  3. 2024Agent S: An Open Agentic Framework that Uses Computers Like a Humancited by 187
  4. 2025Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agentscited by 159
  5. 2024Aria-UI: Visual Grounding for GUI Instructionscited by 140
  6. 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesiscited by 139
  7. 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 81

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 40; 2026: 66.

106 papers.

Sort:

By area:

  1. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  2. 2025Mobile-Agent-v3: Fundamental Agents for GUI Automationcited by 190
  3. 2025MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Serverscited by 96
  4. 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agentscited by 91
  5. 2026Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agentscited by 79
  6. 2025ScaleCUA: Scaling Open-Source Computer Use Agents with Cross-Platform Datacited by 60
  7. 2025CoAct-1: Computer-using Multi-Agent System with Coding Actionscited by 35
  8. 2025UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Actioncited by 31
  9. 2026LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decodingcited by 28
  10. 2025InfiGUI-G1: Advancing GUI Grounding with Adaptive Exploration Policy Optimizationcited by 27
  11. 2025Scaling Synthetic Task Generation for Agents via Explorationcited by 23
  12. 2025UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoningcited by 22
38 more
  1. 2025Grounding Computer Use Agents on Human Demonstrationscited by 22
  2. 2026OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agentcited by 21
  3. 2025CUARewardBench: A Benchmark for Evaluating Reward Models on Computer-using Agentcited by 18
  4. 2025MGA: Memory-Driven GUI Agent for Observation-Centric Interactioncited by 18
  5. 2025Ferret-UI Lite: Lessons from Building Small On-Device GUI Agentscited by 16
  6. 2026GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RLcited by 16
  7. 2025VeriOS: Query-Driven Proactive Human-Agent-GUI Interaction for Trustworthy OS Agentscited by 15
  8. 2025SSRL: Self-Search Reinforcement Learningcited by 14
  9. 2025GUI-ReWalk: Massive Data Generation for GUI Agent via Stochastic Exploration and Intent-Aware Reasoningcited by 13
  10. 2026OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task Executioncited by 13
  11. 2025Learning GUI Grounding with Spatial Reasoning from Visual Feedbackcited by 12
  12. 2025R-WoM: Retrieval-augmented World Model For Computer-use Agentscited by 12
  13. 2026Agent Alpha: Tree Search Unifying Generation, Exploration and Evaluation for Computer-Use Agentscited by 12
  14. 2026The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agentscited by 12
  15. 2025Evolving in Tasks: Empowering the Multi-modality Large Language Model as the Computer Use Agentcited by 11
  16. 2026Learn where to Click from Yourself: On-Policy Self-Distillation for GUI Groundingcited by 11
  17. 2026ToolCUA: Towards Optimal GUI-Tool Path Orchestration for Computer Use Agentscited by 11
  18. 2026Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agentscited by 11
  19. 2025textscGUI-Spotlight: Adaptive Iterative Focus Refinement for Enhanced GUI Visual Groundingcited by 10
  20. 2025Reinforcement Learning for Large Model: A Surveycited by 9
  21. 2025Agentic Lybic: Multi-Agent Execution System with Tiered Reasoning and Orchestrationcited by 8
  22. 2025GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Groundingcited by 8
  23. 2026ShowUI-Aloha: Human-Taught GUI Agentcited by 8
  24. 2026UI-Zoomer: Uncertainty-Driven Adaptive Zoom-In for GUI Groundingcited by 8
  25. 2025SCUBA: Salesforce Computer Use Benchmarkcited by 7
  26. 2025Chain-of-Ground: Improving GUI Grounding via Iterative Reasoning and Reference Feedbackcited by 7
  27. 2025VenusBench-GD: A Comprehensive Multi-Platform GUI Benchmark for Diverse Grounding Taskscited by 7
  28. 2026Video2GUI: Synthesizing Large-Scale Interaction Trajectories for Generalized GUI Agent Pretrainingcited by 7
  29. 2026SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RLcited by 7
  30. 2025SAGE: Training Smart Any-Horizon Agents for Long Video Reasoning with Reinforcement Learningcited by 6
  31. 2025OS-Oracle: A Comprehensive Framework for Cross-Platform GUI Critic Modelscited by 6
  32. 2026AdaZoom-GUI: Adaptive Zoom-based GUI Grounding with Instruction Refinementcited by 6
  33. 2026Agent Planning Benchmark: A Diagnostic Framework for Planning Capabilities in LLM Agentscited by 6
  34. 2025WebDART: Dynamic Decomposition and Re-planning for Complex Web Taskscited by 5
  35. 2025Automated Penetration Testing with LLM Agents and Classical Planningcited by 5
  36. 2026Zoom to Essence: Trainless GUI Grounding by Inferring upon Interface Elementscited by 5
  37. 2026Measure Twice, Click Once: Co-evolving Proposer and Visual Critic via Reinforcement Learning for GUI Groundingcited by 5
  38. 2026VLAA-GUI: Knowing When to Stop, Recover, and Search, A Modular Framework for GUI Automationcited by 5

50 most cited of 106 All 106 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025UI-AGILE: Advancing GUI Agents with Effective Reinforcement Learning and Precise Inference-Time Grounding
  2. 2025GUI-Bee: Align GUI Action Grounding to Novel Environments via Autonomous Exploration
  3. 2025Test-Time Reinforcement Learning for GUI Grounding via Region Consistency
  4. 2025Visual Test-time Scaling for GUI Agent Grounding
  5. 2025ScaleTrack: Scaling and back-tracking Automated GUI Agents
  6. 2026Scaling GUI Agents with Visual State Transitions
  7. 2026Learn from Move: the Next Step for GUI Agents
  8. 2026Test-Time Self-Evolving GUI Visual Grounding via Reflection-Guided On-Policy Self-Distillation