COME: Adding Scene-Centric Forecasting Control to Occupancy World Model
On the map: World models → World models for autonomous driving
Abstract (arXiv)
World models are critical for autonomous driving to simulate environmental dynamics and generate synthetic data. Existing methods struggle to disentangle ego-vehicle motion (perspective shifts) from scene evolvement (agent interactions), leading to suboptimal predictions. Instead, we propose to separate environmental changes from ego-motion by leveraging the scene-centric coordinate systems. In this paper, we introduce COME: a framework that integrates scene-centric forecasting Control into the Occupancy world ModEl. Specifically, COME first generates ego-irrelevant, spatially consistent future features through a scene-centric prediction branch, which are then converted into scene condition using a tailored ControlNet. These condition features are subsequently injected into the occupancy world model, enabling more accurate and controllable future occupancy predictions. Experimental results on the nuScenes-Occ3D dataset show that COME achieves consistent and significant improvements over state-of-the-art (SOTA) methods across diverse configurations, including different input sources (ground-truth, camera-based, fusion-based occupancy) and prediction horizons (3s and 8s). For example, under the same settings, COME achieves 26.3% better mIoU metric than DOME and 23.7% better mIoU metric than UniScene. These results highlight the efficacy of disentangled representation learning in enhancing spatio-temporal prediction fidelity for world models. Code and videos will be available at https://github.com/synsin0/COME.
Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 31 of its 37 references that are in the map. First posted on arXiv 16 Jun 2025.
Builds on
31 of 37 references.
Closest ancestors · 13
Same direction, newest first
- 2024An Efficient Occupancy World Model via Decoupled Dynamic Flow and Image-assisted Training18 Dec
- 2024DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Model14 Oct
- 2024RenderWorld: World Model with Self-Supervised 3D Label17 Sep
- 2024OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Driving5 Sep
- 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving26 Aug
- 2024UnO: Unsupervised Occupancy Fields for Perception and Forecasting12 Jun
- 2023Visual Point Cloud Forecasting enables Scalable Autonomous Driving29 Dec
- 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
- 2023Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications29 Nov
- 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving27 Nov
- 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
- 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving27 Apr
1 more
Other references · 18
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2017Neural Discrete Representation Learningcited by 4,712
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
- 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
- 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
- 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
- 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 298
6 more
- 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
- 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 106
- 2024LidarDM: Generative LiDAR Simulation in a Generated Worldcited by 54
- 2024DrivingSphere: Building a High-fidelity 4D World for Closed-loop Simulationcited by 34
- 2025Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Modelscited by 28
- 2023LiDAR-based 4D Occupancy Completion and Forecastingcited by 23
Built on by
Citing papers per year: 2025: 7; 2026: 7.
14 papers.
Sort:
By area:
- 2025A Comprehensive Survey on World Models for Embodied AIcited by 68
- 20253D and 4D World Modeling: A Surveycited by 67
- 2025AD-R1: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving with Impartial World Modelscited by 14
- 2025GenieDrive: Towards Physics-Aware Driving World Model with 4D Occupancy Guided Video Generationcited by 11
- 2025SparseWorld-TC: Trajectory-Conditioned Sparse Occupancy World Modelcited by 4
- 2026GEM: Generating LiDAR World Model via Deformable Mambacited by 1
- 2025SuperQuadricOcc: Real-Time Self-Supervised Semantic Occupancy Estimation with Superquadric Volume Renderingcited by 0
- 2025OccStress: Stress-Testing the 4D Occupancy Forecasting Chaincited by 0
- 2026OccSim: Multi-kilometer Simulation with Long-horizon Occupancy World Modelscited by 0
- 2026OccDirector: Language-Guided Behavior and Interaction Generation in 4D Occupancy Spacecited by 0
- 2026GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planningcited by 0
- 2026LiAuto-GeoX: Efficient Grounded Driving Transformercited by 0
- 2026Representation Learning in Diffusion and Flow-based Model: An Application Aspect25 Aug 2026
- 2026Geometry-Aware Spatio-Temporal Context Modeling for 4D Occupancy Forecasting15 Aug 2026
- 2026LiAuto-GeoX: Efficient Grounded Driving Transformer4 Jun 2026
- 2026GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planning17 May 2026
- 2026GEM: Generating LiDAR World Model via Deformable Mamba8 May 2026
- 2026OccDirector: Language-Guided Behavior and Interaction Generation in 4D Occupancy Space24 Apr 2026
- 2026OccSim: Multi-kilometer Simulation with Long-horizon Occupancy World Models30 Mar 2026
- 2025OccStress: Stress-Testing the 4D Occupancy Forecasting Chain17 Dec 2025
- 2025GenieDrive: Towards Physics-Aware Driving World Model with 4D Occupancy Guided Video Generation14 Dec 2025
- 2025SparseWorld-TC: Trajectory-Conditioned Sparse Occupancy World Model27 Nov 2025
- 2025AD-R1: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving with Impartial World Models25 Nov 2025
- 2025SuperQuadricOcc: Real-Time Self-Supervised Semantic Occupancy Estimation with Superquadric Volume Rendering21 Nov 2025
2 more
- 2025A Comprehensive Survey on World Models for Embodied AI19 Oct 2025
- 20253D and 4D World Modeling: A Survey4 Sep 2025
- 2025A Comprehensive Survey on World Models for Embodied AIcited by 68
- 20253D and 4D World Modeling: A Surveycited by 67
- 2025AD-R1: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving with Impartial World Modelscited by 14
- 2025GenieDrive: Towards Physics-Aware Driving World Model with 4D Occupancy Guided Video Generationcited by 11
- 2025SparseWorld-TC: Trajectory-Conditioned Sparse Occupancy World Modelcited by 4
- 2026GEM: Generating LiDAR World Model via Deformable Mambacited by 1
- 2026OccSim: Multi-kilometer Simulation with Long-horizon Occupancy World Modelscited by 0
- 2026OccDirector: Language-Guided Behavior and Interaction Generation in 4D Occupancy Spacecited by 0
- 2026GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planningcited by 0
- 2026GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planning17 May 2026
- 2026GEM: Generating LiDAR World Model via Deformable Mamba8 May 2026
- 2026OccDirector: Language-Guided Behavior and Interaction Generation in 4D Occupancy Space24 Apr 2026
- 2026OccSim: Multi-kilometer Simulation with Long-horizon Occupancy World Models30 Mar 2026
- 2025GenieDrive: Towards Physics-Aware Driving World Model with 4D Occupancy Guided Video Generation14 Dec 2025
- 2025SparseWorld-TC: Trajectory-Conditioned Sparse Occupancy World Model27 Nov 2025
- 2025AD-R1: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving with Impartial World Models25 Nov 2025
- 2025A Comprehensive Survey on World Models for Embodied AI19 Oct 2025
- 20253D and 4D World Modeling: A Survey4 Sep 2025
- 2025SuperQuadricOcc: Real-Time Self-Supervised Semantic Occupancy Estimation with Superquadric Volume Renderingcited by 0
- 2025OccStress: Stress-Testing the 4D Occupancy Forecasting Chaincited by 0
- 2026LiAuto-GeoX: Efficient Grounded Driving Transformercited by 0
- 2026Geometry-Aware Spatio-Temporal Context Modeling for 4D Occupancy Forecastingcited by 0
- 2026Geometry-Aware Spatio-Temporal Context Modeling for 4D Occupancy Forecasting15 Aug 2026
- 2026LiAuto-GeoX: Efficient Grounded Driving Transformer4 Jun 2026
- 2025OccStress: Stress-Testing the 4D Occupancy Forecasting Chain17 Dec 2025
- 2025SuperQuadricOcc: Real-Time Self-Supervised Semantic Occupancy Estimation with Superquadric Volume Rendering21 Nov 2025
Similar, not cited
Close in wording, no citation either way
- 2026OccWAM: Occupancy World-Action Models with Joint Scene-to-Ego Flow Matching
- 2025Vision-Centric 4D Occupancy Forecasting and Planning via Implicit Residual World Models
- 2025OccTENS: 3D Occupancy World Model via Temporal Next-Scale Prediction
- 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Driving
- 2025A Survey of World Models for Autonomous Driving
- 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Prediction
- 2025Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving
- 2026CascadeOcc: Rethinking 3D Occupancy World Models with Cascaded VQ Representations