# ViMo: A Generative Visual GUI World Model for App Agents

arXiv: https://arxiv.org/abs/2504.13936 · HTML: https://mlsanity.com/paper/2504.13936/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Placed in World model security and GUI agents, one of 11 directions in World models.
- Builds on 44 of its 68 references that are in the map.
- First posted on arXiv 15 Apr 2025.

## Placed in

- World models / [World model security and GUI agents](https://mlsanity.com/wm/world-model-security-and-gui-agents/)

## Builds on (most cited first, 30 of 44)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents](https://mlsanity.com/paper/2405.14573/) (2024)
- [AppAgent: Multimodal Agents as Smartphone Users](https://mlsanity.com/paper/2312.13771/) (2023)
- [Android in the Wild: A Large-Scale Dataset for Android Device Control](https://mlsanity.com/paper/2307.10088/) (2023)
- [Diffusion Models Are Real-Time Game Engines](https://mlsanity.com/paper/2408.14837/) (2024)
- [ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving](https://mlsanity.com/paper/2309.17452/) (2023)
- [LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation](https://mlsanity.com/paper/2303.17189/) (2023)
- [The Creation and Detection of Deepfakes: A Survey](https://mlsanity.com/paper/2004.11138/) (2020)
- [TextDiffuser: Diffusion Models as Text Painters](https://mlsanity.com/paper/2305.10855/) (2023)
- [Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents](https://mlsanity.com/paper/2408.07199/) (2024)
- [Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration](https://mlsanity.com/paper/2406.01014/) (2024)
- [AutoDroid: LLM-powered Task Automation in Android](https://mlsanity.com/paper/2308.15272/) (2023)
- [Understanding World or Predicting Future? A Comprehensive Survey of World Models](https://mlsanity.com/paper/2411.14499/) (2024)
- [Large Language Model-Brained GUI Agents: A Survey](https://mlsanity.com/paper/2411.18279/) (2024)
- [Android in the Zoo: Chain-of-Action-Thought for GUI Agents](https://mlsanity.com/paper/2403.02713/) (2024)
- [Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents](https://mlsanity.com/paper/2411.06559/) (2024)
- [Tree Search for Language Model Agents](https://mlsanity.com/paper/2407.01476/) (2024)
- [TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering](https://mlsanity.com/paper/2311.16465/) (2023)
- [Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation](https://mlsanity.com/paper/2410.13232/) (2024)
- [SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognition](https://mlsanity.com/paper/2005.10977/) (2020)
- [Image Generation from Layout](https://mlsanity.com/paper/1811.11389/) (2018)

## Built on by (most cited first, 30 of 35)

- [A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents](https://mlsanity.com/paper/2601.12538/) (2026)
- [LLM-Powered GUI Agents in Phone Automation: Surveying Progress and Prospects](https://mlsanity.com/paper/2504.19838/) (2025)
- [Code2World: A GUI World Model via Renderable Code Generation](https://mlsanity.com/paper/2602.09856/) (2026)
- [WebWorld: A Large-Scale World Model for Web Agent Training](https://mlsanity.com/paper/2602.14721/) (2026)
- [MobileWorldBench: Towards Semantic World Modeling For Mobile Agents](https://mlsanity.com/paper/2512.14014/) (2025)
- [GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training](https://mlsanity.com/paper/2602.14093/) (2026)
- [Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond](https://mlsanity.com/paper/2604.22748/) (2026)
- [Generative Visual Code Mobile World Models](https://mlsanity.com/paper/2602.01576/) (2026)
- [MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research](https://mlsanity.com/paper/2605.26114/) (2026)
- [How Mobile World Model Guides GUI Agents?](https://mlsanity.com/paper/2605.10347/) (2026)
- [Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application](https://mlsanity.com/paper/2606.12191/) (2026)
- [OccuBench: Evaluating AI Agents on Real-World Professional Tasks via Language Environment Simulation](https://mlsanity.com/paper/2604.10866/) (2026)
- [On the Importance of Task Complexity in Evaluating LLM-Based Multi-Agent Systems](https://mlsanity.com/paper/2510.04311/) (2025)
- [GUI-GenBench: Evaluating Image Generation Models as Interactive GUI Environments](https://mlsanity.com/paper/2602.09007/) (2026)
- [CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use Agents](https://mlsanity.com/paper/2603.24440/) (2026)
- [UI-Oceanus: Scaling GUI Agents with Synthetic Environmental Dynamics](https://mlsanity.com/paper/2604.02345/) (2026)
- [Anticipatory Planning for Multimodal AI Agents](https://mlsanity.com/paper/2603.16777/) (2026)
- [Scaling GUI Agents with Visual State Transitions](https://mlsanity.com/paper/2607.24112/) (2026)
- [Beyond Syntax: Action Semantics Learning for App Agents](https://mlsanity.com/paper/2506.17697/) (2025)
- [SeerGuard: A Safety Framework for Mobile GUI Agents via World Model Prediction](https://mlsanity.com/paper/2607.15550/) (2026)
- [World Models Meet Language Models: On the Complementarity of Concrete and Abstract Reasoning](https://mlsanity.com/paper/2606.03603/) (2026)
- [AppDeltaWorld: Transition-Grounded Delta Code World Model for Mobile GUI Agents](https://mlsanity.com/paper/2608.05891/) (2026)
- [GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments](https://mlsanity.com/paper/2609.00048/) (2026)
- [Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injection](https://mlsanity.com/paper/2604.07831/) (2026)
- [MIRAGE: Mobile Agents with Implicit Reasoning and Generative World Models](https://mlsanity.com/paper/2606.04627/) (2026)
- [Software Engineering for and with GUI Agent](https://mlsanity.com/paper/2608.09278/) (2026)
- [Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability](https://mlsanity.com/paper/2605.07110/) (2026)
- [Beyond Binary: Reframing GUI Critique as Continuous Semantic Alignment](https://mlsanity.com/paper/2605.14311/) (2026)
- [Quo Vadis, World Modeling?](https://mlsanity.com/paper/2608.02713/) (2026)
- [WM-R1: Training GUI Agents to Reason and leverage World Models with Reinforcement Learning](https://mlsanity.com/paper/2608.27508/) (2026)

## Abstract (arXiv)

App agents, which autonomously operate mobile Apps through Graphical User Interfaces (GUIs), have gained significant interest in real-world applications. Yet, they often struggle with long-horizon planning, failing to find the optimal actions for complex tasks with longer steps. To address this, world models are used to predict the next GUI observation based on user actions, enabling more effective agent planning. However, existing world models primarily focus on generating only textual descriptions, lacking essential visual details. To fill this gap, we propose ViMo, the first visual world model designed to generate future App observations as images. For the challenge of generating text in image patches, where even minor pixel errors can distort readability, we decompose GUI generation into graphic and text content generation. We propose a novel data representation, the Symbolic Text Representation~(STR) to overlay text content with symbolic placeholders while preserving graphics. With this design, ViMo employs a STR Predictor to predict future GUIs' graphics and a GUI-text Predictor for generating the corresponding text. Moreover, we deploy ViMo to enhance agent-focused tasks by predicting the outcome of different action options. Experiments show ViMo's ability to generate visually plausible and functionally effective GUIs that enable App agents to make more informed decisions.