mlsanityData to 14 Oct 2026

SEE: Structure-aware Exploring & Exploiting for Long-horizon GUI Agent Trajectory Synthesis

arXiv 2607.18046 ↗First posted 20 Jul 2026cs.LGcited by 1

On the map: LLM agents and tool use → GUI and mobile agents

Abstract (arXiv)

Graphical User Interface (GUI) agents powered by vision-language models hold promise for automating real-world mobile tasks. However, progress is limited by the lack of high-coverage, long-horizon interaction trajectories collected from element-rich and rapidly evolving apps. Existing pipelines often rely on costly human demonstrations or on-policy framework, which tends to over-sample common flows while missing rare transitions and complex multi-step procedures. To address this problem, we propose SEE, a two-stage data synthesis framework consisting of (i) an efficient exploration stage that builds an explicit UI transition graph over screens and elements, and (ii) a graph-based synthesis stage that composes diverse multi-step trajectories via planning and controlled sampling. This design yields reproducible and explainable data generation, while explicitly preventing spurious cycles and enabling long-horizon composition. Across multiple real-world apps, SEE produces trajectories with an average length of 14.8 steps while avoiding spurious loops, and agents fine-tuned on SEE achieve improved task success and generalization to unseen screens. We will publicly release our synthesis code and dataset.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 24 of its 46 references that are in the map. First posted on arXiv 20 Jul 2026.

Builds on

24 of 46 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025GUI-ReWalk: Massive Data Generation for GUI Agent via Stochastic Exploration and Intent-Aware Reasoning19 Sep
  2. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Use6 Aug
  3. 2025UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based Mobile GUI Agents27 May
  4. 2025TongUI: Internet-Scale Trajectories from Multimodal Web Tutorials for Generalized GUI Agents17 Apr
  5. 2025Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks20 Jan
  6. 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis27 Dec
  7. 2024Large Language Model-Brained GUI Agents: A Survey27 Nov
  8. 2024AndroidLab: Training and Systematic Benchmarking of Android Autonomous Agents31 Oct
  9. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
  10. 2024AMEX: Android Multi-annotation Expo Dataset for Mobile GUI Agents3 Jul
  11. 2024On the Effects of Data Scale on UI Control Agents6 Jun
  12. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments11 Apr
1 more
  1. 2023CogAgent: A Visual Language Model for GUI Agents14 Dec

Other references · 11

  1. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  2. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  3. 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 421
  4. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 284
  5. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 278
  6. 2023Auto-GPT for Online Decision Making: Benchmarks and Additional Opinionscited by 255
  7. 2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 194
  8. 2024WebLINX: Real-World Website Navigation with Multi-Turn Dialoguecited by 180
  9. 2020State-Relabeling Adversarial Active Learningcited by 60
  10. 2025WebSynthesis: World-Model-Guided MCTS for Efficient WebUI-Trajectory Synthesiscited by 21
  11. 2024Downstream-Pretext Domain Knowledge Traceback for Active Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026HATS: Hardness-Aware Trajectory Synthesis for GUI Agents
  2. 2025GUI-Xplore: Empowering Generalizable GUI Agents with One Exploration
  3. 2026GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments
  4. 2026UI-KOBE: Knowledge-Oriented Behavior Exploration for Lightweight Graph-Guided GUI Agents
  5. 2025GUI-explorer: Autonomous Exploration and Mining of Transition-aware Knowledge for GUI Agent
  6. 2026A Task-State Representation for Long-Horizon Mobile GUI Agents
  7. 2026Towards Omni-dimensional GUI Agent Navigation with Masked Trajectory Prediction
  8. 2026ANCHOR: Branch-Point Data Generation for GUI Agents