X-World: Controllable Ego-Centric Multi-Camera World Models for Scalable End-to-End Driving
On the map: World models → World models for autonomous driving
also in 1 more
Abstract (arXiv)
Scalable and reliable evaluation is increasingly critical in the end-to-end era of autonomous driving, where vision--language--action (VLA) policies directly map raw sensor streams to driving actions. Yet, current evaluation pipelines still rely heavily on real-world road testing, which is costly, biased toward limited scenario coverage, and difficult to reproduce. These challenges motivate a real-world simulator that can generate realistic future observations under proposed actions, while remaining controllable and stable over long horizons. We present X-World, an action-conditioned multi-camera generative world model that simulates future observations directly in video space. Given synchronized multi-view camera history and a future action sequence, X-World generates future multi-camera video streams that follow the commanded actions. To ensure reproducible and editable scene rollouts, X-World further supports optional controls over dynamic traffic agents and static road elements, and retains a text-prompt interface for appearance-level control (e.g., weather and time of day). Beyond world simulation, X-World also enables video style transfer by conditioning on appearance prompts while preserving the underlying action and scene dynamics. At the core of X-World is a multi-view latent video generator designed to explicitly encourage cross-view geometric consistency and temporal coherence under diverse control signals. Experiments show that X-World achieves high-quality multi-view video generation with (i) strong view consistency across cameras, (ii) stable temporal dynamics over long rollouts, and (iii) high controllability with strict action following and faithful adherence to optional scene controls. These properties make X-World a practical foundation for scalable and reproducible evaluation.
Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 10 of its 14 references that are in the map. First posted on arXiv 20 Mar 2026.
Builds on
10 of 14 references.
References · 10
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
- 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 684
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
- 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 231
- 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 166
- 2023UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretrainingcited by 151
Built on by
Citing papers per year: 2026: 8.
8 papers.
- 2026DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Drivingcited by 14
- 2026M4World: A Multi-view Multimodal Driving World Model for Interactive Object Manipulation and Minute-long Streamingcited by 2
- 2026X-Foresight: A Joint Vision-Action Causal Forecasting Network via Predictive World Modelingcited by 1
- 2026Multi-Agent Embodied Autonomous Driving (MAEAD): From V2X Information Exchange to Shared World Modelscited by 0
- 2026ZYT-World: A Real-Time Controllable World Model for Closed-Loop Autonomous-Driving Simulationcited by 0
- 2026Action Forcing: Training World Models on Unsupervised Video by Recovering Underlying Egomotion Basescited by 0
- 2026RoXDrive: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving via Action-Faithful Rolloutscited by 0
- 2026Connected Self Forcing: Beyond Local Learning in Video Autoregressioncited by 0
- 2026Connected Self Forcing: Beyond Local Learning in Video Autoregression8 Oct 2026
- 2026RoXDrive: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving via Action-Faithful Rollouts29 Sep 2026
- 2026Action Forcing: Training World Models on Unsupervised Video by Recovering Underlying Egomotion Bases24 Sep 2026
- 2026ZYT-World: A Real-Time Controllable World Model for Closed-Loop Autonomous-Driving Simulation18 Sep 2026
- 2026M4World: A Multi-view Multimodal Driving World Model for Interactive Object Manipulation and Minute-long Streaming15 Jul 2026
- 2026Multi-Agent Embodied Autonomous Driving (MAEAD): From V2X Information Exchange to Shared World Models11 Jun 2026
- 2026DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving27 May 2026
- 2026X-Foresight: A Joint Vision-Action Causal Forecasting Network via Predictive World Modeling24 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving
- 2026HelloWorld: Towards Practical Applications of Generative Driving World Models
- 2025ProphetDWM: A Driving World Model for Rolling Out Future Actions and Videos
- 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving
- 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation
- 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving
- 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT
- 2025SimWorld: A Unified Benchmark for Simulator-Conditioned Scene Generation via World Model