mlsanityData to 14 Oct 2026

COME: Adding Scene-Centric Forecasting Control to Occupancy World Model

arXiv 2506.13260 ↗First posted 16 Jun 2025cs.CVcited by 14

On the map: World models → World models for autonomous driving

Abstract (arXiv)

World models are critical for autonomous driving to simulate environmental dynamics and generate synthetic data. Existing methods struggle to disentangle ego-vehicle motion (perspective shifts) from scene evolvement (agent interactions), leading to suboptimal predictions. Instead, we propose to separate environmental changes from ego-motion by leveraging the scene-centric coordinate systems. In this paper, we introduce COME: a framework that integrates scene-centric forecasting Control into the Occupancy world ModEl. Specifically, COME first generates ego-irrelevant, spatially consistent future features through a scene-centric prediction branch, which are then converted into scene condition using a tailored ControlNet. These condition features are subsequently injected into the occupancy world model, enabling more accurate and controllable future occupancy predictions. Experimental results on the nuScenes-Occ3D dataset show that COME achieves consistent and significant improvements over state-of-the-art (SOTA) methods across diverse configurations, including different input sources (ground-truth, camera-based, fusion-based occupancy) and prediction horizons (3s and 8s). For example, under the same settings, COME achieves 26.3% better mIoU metric than DOME and 23.7% better mIoU metric than UniScene. These results highlight the efficacy of disentangled representation learning in enhancing spatio-temporal prediction fidelity for world models. Code and videos will be available at https://github.com/synsin0/COME.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 31 of its 37 references that are in the map. First posted on arXiv 16 Jun 2025.

Builds on

31 of 37 references.

Closest ancestors · 13

Same direction, newest first

  1. 2024An Efficient Occupancy World Model via Decoupled Dynamic Flow and Image-assisted Training18 Dec
  2. 2024DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Model14 Oct
  3. 2024RenderWorld: World Model with Self-Supervised 3D Label17 Sep
  4. 2024OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Driving5 Sep
  5. 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving26 Aug
  6. 2024UnO: Unsupervised Occupancy Fields for Perception and Forecasting12 Jun
  7. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Driving29 Dec
  8. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
  9. 2023Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications29 Nov
  10. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving27 Nov
  11. 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
  12. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving27 Apr
1 more
  1. 2023Point Cloud Forecasting as a Proxy for 4D Occupancy Forecasting25 Feb

Other references · 18

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2022Scalable Diffusion Models with Transformerscited by 6,054
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  6. 2017Neural Discrete Representation Learningcited by 4,712
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  8. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  9. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  10. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  11. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  12. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 298
6 more
  1. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  2. 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 106
  3. 2024LidarDM: Generative LiDAR Simulation in a Generated Worldcited by 54
  4. 2024DrivingSphere: Building a High-fidelity 4D World for Closed-loop Simulationcited by 34
  5. 2025Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Modelscited by 28
  6. 2023LiDAR-based 4D Occupancy Completion and Forecastingcited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 7.

14 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2026OccWAM: Occupancy World-Action Models with Joint Scene-to-Ego Flow Matching
  2. 2025Vision-Centric 4D Occupancy Forecasting and Planning via Implicit Residual World Models
  3. 2025OccTENS: 3D Occupancy World Model via Temporal Next-Scale Prediction
  4. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Driving
  5. 2025A Survey of World Models for Autonomous Driving
  6. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Prediction
  7. 2025Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving
  8. 2026CascadeOcc: Rethinking 3D Occupancy World Models with Cascaded VQ Representations