# Code2World: A GUI World Model via Renderable Code Generation

arXiv: https://arxiv.org/abs/2602.09856 · HTML: https://mlsanity.com/paper/2602.09856/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in World model security and GUI agents, one of 11 directions in World models.
- Builds on 50 of its 53 references that are in the map.
- First posted on arXiv 10 Feb 2026.

## Placed in

- World models / [World model security and GUI agents](https://mlsanity.com/wm/world-model-security-and-gui-agents/)
- LLM agents and tool use / [GUI and mobile agents](https://mlsanity.com/agents/gui-and-mobile-agents/)

## Builds on (most cited first, 30 of 50)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://mlsanity.com/paper/2501.17811/) (2025)
- [CogAgent: A Visual Language Model for GUI Agents](https://mlsanity.com/paper/2312.08914/) (2023)
- [UI-TARS: Pioneering Automated GUI Interaction with Native Agents](https://mlsanity.com/paper/2501.12326/) (2025)
- [AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents](https://mlsanity.com/paper/2405.14573/) (2024)
- [GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning](https://mlsanity.com/paper/2507.01006/) (2025)
- [GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents](https://mlsanity.com/paper/2504.10458/) (2025)
- [ShowUI: One Vision-Language-Action Model for GUI Visual Agent](https://mlsanity.com/paper/2411.17465/) (2024)
- [Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers](https://mlsanity.com/paper/2506.23918/) (2025)
- [On the Effects of Data Scale on UI Control Agents](https://mlsanity.com/paper/2406.03679/) (2024)
- [Mobile-Agent-v3: Fundamental Agents for GUI Automation](https://mlsanity.com/paper/2508.15144/) (2025)
- [UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning](https://mlsanity.com/paper/2503.21620/) (2025)
- [Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents](https://mlsanity.com/paper/2411.06559/) (2024)
- [GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoning](https://mlsanity.com/paper/2504.02546/) (2025)
- [InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners](https://mlsanity.com/paper/2504.14239/) (2025)
- [Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation](https://mlsanity.com/paper/2410.13232/) (2024)
- [Tree Search for LLM Agent Reinforcement Learning](https://mlsanity.com/paper/2509.21240/) (2025)
- [UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interaction](https://mlsanity.com/paper/2503.15661/) (2025)
- [ARPO:End-to-End Policy Optimization for GUI Agents with Experience Replay](https://mlsanity.com/paper/2505.16282/) (2025)
- [MobileGUI-RL: Advancing Mobile GUI Agent through Reinforcement Learning in Online Environment](https://mlsanity.com/paper/2507.05720/) (2025)
- [GWM: Towards Scalable Gaussian World Models for Robotic Manipulation](https://mlsanity.com/paper/2508.17600/) (2025)

## Built on by (most cited first, 29 of 29)

- [Code as Agent Harness](https://mlsanity.com/paper/2605.18747/) (2026)
- [Qwen-AgentWorld: Language World Models for General Agents](https://mlsanity.com/paper/2606.24597/) (2026)
- [Omni-WorldBench: Towards a Comprehensive Interaction-Centric Evaluation for World Models](https://mlsanity.com/paper/2603.22212/) (2026)
- [ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agents](https://mlsanity.com/paper/2604.11784/) (2026)
- [Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond](https://mlsanity.com/paper/2604.22748/) (2026)
- [LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks](https://mlsanity.com/paper/2608.01964/) (2026)
- [How Mobile World Model Guides GUI Agents?](https://mlsanity.com/paper/2605.10347/) (2026)
- [Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application](https://mlsanity.com/paper/2606.12191/) (2026)
- [PatchWorld: Gradient-Free Optimization of Executable World Models for Agent Environments](https://mlsanity.com/paper/2605.30880/) (2026)
- [CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use Agents](https://mlsanity.com/paper/2603.24440/) (2026)
- [Bridging the Agent-World Gap: Text World Models for LLM-based Agents](https://mlsanity.com/paper/2606.09032/) (2026)
- [Learning Agentic Policy from Action Guidance](https://mlsanity.com/paper/2605.12004/) (2026)
- [Scaling GUI Agents with Visual State Transitions](https://mlsanity.com/paper/2607.24112/) (2026)
- [GUI Agents with Reinforcement Learning: Toward Digital Inhabitants](https://mlsanity.com/paper/2604.27955/) (2026)
- [LoopArena: Benchmarking Models as Runtime Controllers for Loop Engineering](https://mlsanity.com/paper/2608.28281/) (2026)
- [AppDeltaWorld: Transition-Grounded Delta Code World Model for Mobile GUI Agents](https://mlsanity.com/paper/2608.05891/) (2026)
- [GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments](https://mlsanity.com/paper/2609.00048/) (2026)
- [Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injection](https://mlsanity.com/paper/2604.07831/) (2026)
- [A History-Aware Visually Grounded Critic for Computer Use Agents](https://mlsanity.com/paper/2606.11078/) (2026)
- [A Survey on Foundations and Frontiers of Multimodal Agentic Frameworks: Techniques and Applications](https://mlsanity.com/paper/2608.20379/) (2026)
- [Object-Centric Environment Modeling for Agentic Tasks](https://mlsanity.com/paper/2607.02846/) (2026)
- [Quo Vadis, World Modeling?](https://mlsanity.com/paper/2608.02713/) (2026)
- [WM-R1: Training GUI Agents to Reason and leverage World Models with Reinforcement Learning](https://mlsanity.com/paper/2608.27508/) (2026)
- [The GUI Is Not the State: Diagnosing State Aliasing in GUI World Models](https://mlsanity.com/paper/2609.32679/) (2026)
- [AutoGUIWorld: Image Generators as Visual World Models for GUI Agent](https://mlsanity.com/paper/2610.01215/) (2026)
- [Right Screen, Wrong Transition: World Models as Verifiers for GUI Agents](https://mlsanity.com/paper/2610.11942/) (2026)
- [Latent World Action Model for GUI Agents](https://mlsanity.com/paper/2610.14228/) (2026)
- [Urbex: Agentic Spatial Grounding for City-Scale 3D Scenes](https://mlsanity.com/paper/2610.18356/) (2026)
- [PrismaCoder: Pushing the Boundaries of Open Multimodal Code Intelligence through Decomposed Rubric Synthesis](https://mlsanity.com/paper/2610.20538/) (2026)

## Abstract (arXiv)

Autonomous GUI agents interact with environments by perceiving interfaces and executing actions. As a virtual sandbox, the GUI World model empowers agents with human-like foresight by enabling action-conditioned prediction. However, existing text- and pixel-based approaches struggle to simultaneously achieve high visual fidelity and fine-grained structural controllability. To this end, we propose Code2World, a vision-language coder that simulates the next visual state via renderable code generation. Specifically, to address the data scarcity problem, we construct AndroidCode by translating GUI trajectories into high-fidelity HTML and refining synthesized code through a visual-feedback revision mechanism, yielding a corpus of over 80K high-quality screen-action pairs. To adapt existing VLMs into code prediction, we first perform SFT as a cold start for format layout following, then further apply Render-Aware Reinforcement Learning which uses rendered outcome as the reward signal by enforcing visual semantic fidelity and action consistency. Extensive experiments demonstrate that Code2World-8B achieves the top-performing next UI prediction, rivaling the competitive GPT-5 and Gemini-3-Pro-Image. Notably, Code2World significantly enhances downstream navigation success rates in a flexible manner, boosting Gemini-2.5-Flash by +9.5% on AndroidWorld navigation. The code is available at https://github.com/AMAP-ML/Code2World.