mlsanityData to 14 Oct 2026

UI-Oceanus: Scaling GUI Agents with Synthetic Environmental Dynamics

arXiv 2604.02345 ↗First posted 11 Feb 2026cs.LG cs.AIcited by 5

On the map: LLM agents and tool use → GUI and mobile agents

also in 1 more
Abstract (arXiv)

Scaling generalist GUI agents is hindered by the data scalability bottleneck of expensive human demonstrations and the "distillation ceiling" of synthetic teacher supervision. To transcend these limitations, we propose UI-Oceanus, a framework that shifts the learning focus from mimicking high-level trajectories to mastering interaction physics via ground-truth environmental feedback. Through a systematic investigation of self-supervised objectives, we identify that forward dynamics, defined as the generative prediction of future interface states, acts as the primary driver for scalability and significantly outweighs inverse inference. UI-Oceanus leverages this insight by converting low-cost autonomous exploration, which is verified directly by system execution, into high-density generative supervision to construct a robust internal world model. Experimental evaluations across a series of models demonstrate the decisive superiority of our approach: models utilizing Continual Pre-Training (CPT) on synthetic dynamics outperform non-CPT baselines with an average success rate improvement of 7% on offline benchmarks, which amplifies to a 16.8% gain in real-world online navigation. Furthermore, we observe that navigation performance scales with synthetic data volume. These results confirm that grounding agents in forward predictive modeling offers a superior pathway to scalable GUI automation with robust cross-domain adaptability and compositional generalization.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 40 of its 53 references that are in the map. First posted on arXiv 11 Feb 2026.

Builds on

40 of 53 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025MobileWorldBench: Towards Semantic World Modeling For Mobile Agents16 Dec
  2. 2025GUI-ReWalk: Massive Data Generation for GUI Agent via Stochastic Exploration and Intent-Aware Reasoning19 Sep
  3. 2025MobileRL: Online Agentic Reinforcement Learning for Mobile GUI Agents10 Sep
  4. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning2 Sep
  5. 2025Mobile-Agent-v3: Fundamental Agents for GUI Automation21 Aug
  6. 2025MagicGUI: A Foundational Mobile GUI Agent with Scalable Data Pipeline and Reinforcement Fine-tuning19 Jul
  7. 2025ProgRM: Build Better GUI Agents with Progress Rewards23 May
  8. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents21 Jan
  9. 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis27 Dec
  10. 2024GUI Agents with Foundation Models: A Comprehensive Survey7 Nov
  11. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
  12. 2024GUICourse: From General Vision Language Models to Versatile GUI Agents17 Jun
4 more
  1. 2024On the Effects of Data Scale on UI Control Agents6 Jun
  2. 2024Android in the Zoo: Chain-of-Action-Thought for GUI Agents5 Mar
  3. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents17 Jan
  4. 2023CogAgent: A Visual Language Model for GUI Agents14 Dec

Other references · 24

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  5. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  6. 2020Knowledge Distillation: A Surveycited by 1,603
  7. 2018World Modelscited by 1,513
  8. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  9. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  10. 2018Behavioral Cloning from Observationcited by 492
  11. 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 319
  12. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 278
12 more
  1. 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 232
  2. 2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 194
  3. 2020Widget Captioning: Generating Natural Language Description for Mobile User Interface Elementscited by 95
  4. 2025Agent Learning via Early Experiencecited by 85
  5. 2025Explorer: Scaling Exploration-driven Web Trajectory Synthesis for Multimodal Web Agentscited by 72
  6. 2025On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Modelscited by 69
  7. 2025VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agentscited by 62
  8. 2025ViMo: A Generative Visual GUI World Model for App Agentscited by 35
  9. 2025Bee: A High-Quality Corpus and Full-Stack Suite to Unlock Advanced Fully Open MLLMscited by 27
  10. 2025Scaling Synthetic Task Generation for Agents via Explorationcited by 23
  11. 2025Rethinking the Role of Text Complexity in Language Model Pretrainingcited by 2
  12. 2024Skill-Adpative Imitation Learning for UI Test Reusecited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training
  2. 2026OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task Execution
  3. 2026Learn from Move: the Next Step for GUI Agents
  4. 2026GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments
  5. 2026UI-Venus-1.5 Technical Report
  6. 2026Benchmarking and Improving GUI Agents in High-Dynamic Environments
  7. 2025GUI-Bee: Align GUI Action Grounding to Novel Environments via Autonomous Exploration
  8. 2026ScaleWoB: Guiding GUI Agents with Coding Agents via Large-Scale Environmental Synthesis