mlsanityData to 7 Oct 2026

Solaris: Towards Interfaces That Are Generated, Not Coded

arXiv 2609.00776 ↗First posted 1 Sep 2026cs.CV cs.AIcited by 0

On the map: World models → World model security and GUI agents

Abstract (arXiv)

Digital interfaces are traditionally implemented through intermediate representations such as code, requiring their appearance and behavior to be specified in advance. We introduce Solaris, an interface world model that instead generates an interactive UI directly, frame by frame, in response to user actions. Solaris treats mouse interactions as conditioning signals and autoregressively synthesizes the resulting visual state at interactive speeds. To enable real-time generation while maintaining visual coherence over extended interactions, we combine autoregressive frame generation with few-step distillation and training on the model's own outputs. A language model complements the visual world model by interpreting user intent and specifying how interactions should affect the generated environment, separating high-level reasoning from visual rendering. By generating both the appearance and behavior of an interface dynamically, Solaris enables open-ended interactions that need not be explicitly programmed in advance. We view interface world models as a step toward a new paradigm for software, where interfaces are generated and adapted continuously around user intent rather than implemented as fixed collections of predefined states and

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in World model security and GUI agents, one of 11 directions in World models. Builds on 5 of its 9 references that are in the map. First posted on arXiv 1 Sep 2026.

Builds on

5 of 9 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025ViMo: A Generative Visual GUI World Model for App Agents15 Apr

Other references · 4

  1. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  2. 2026OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Taskscited by 29
  3. 2025NeuralOS: Towards Simulating Operating Systems via Neural Generative Modelscited by 17
  4. 2026Computer-Using World Modelcited by 16

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026AutoGUI-v2: A Comprehensive Multi-Modal GUI Functionality Understanding Benchmark
  2. 2026UI2App: Benchmarking Visual Interaction Inference in Executable Web Application Generation
  3. 2026AutoGUIWorld: Image Generators as Visual World Models for GUI Agent
  4. 2025Computer-Use Agents as Judges for Generative User Interface
  5. 2025ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agents
  6. 2026Code2World: A GUI World Model via Renderable Code Generation
  7. 2026Benchmarking and Improving GUI Agents in High-Dynamic Environments
  8. 2025UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interaction