# Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving

arXiv: https://arxiv.org/abs/2502.07309 · HTML: https://mlsanity.com/paper/2502.07309/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in World models for autonomous driving, one of 11 directions in World models.
- Builds on 15 of its 44 references that are in the map.
- First posted on arXiv 11 Feb 2025.

## Placed in

- World models / [World models for autonomous driving](https://mlsanity.com/wm/world-models-for-autonomous-driving/)
- End-to-end autonomous driving / [Generative simulation for driving](https://mlsanity.com/driving/generative-simulation-for-driving/)

## Builds on (most cited first, 15 of 15)

- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [GAIA-1: A Generative World Model for Autonomous Driving](https://mlsanity.com/paper/2309.17080/) (2023)
- [MagicDrive: Street View Generation with Diverse 3D Geometry Control](https://mlsanity.com/paper/2310.02601/) (2023)
- [OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving](https://mlsanity.com/paper/2311.16038/) (2023)
- [Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenes](https://mlsanity.com/paper/2305.10430/) (2023)
- [FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformation](https://mlsanity.com/paper/2307.01492/) (2023)
- [Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion](https://mlsanity.com/paper/2311.01017/) (2023)
- [AMVNet: Assertion-based Multi-View Fusion Network for LiDAR Semantic Segmentation](https://mlsanity.com/paper/2012.04934/) (2020)
- [OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Driving](https://mlsanity.com/paper/2409.03272/) (2024)
- [Fully Sparse 3D Occupancy Prediction](https://mlsanity.com/paper/2312.17118/) (2023)
- [MonoOcc: Digging into Monocular Semantic Occupancy Prediction](https://mlsanity.com/paper/2403.08766/) (2024)
- [OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environments](https://mlsanity.com/paper/2312.09243/) (2023)
- [DRINet++: Efficient Voxel-as-point Point Cloud Segmentation](https://mlsanity.com/paper/2111.08318/) (2021)
- [OccFlowNet: Towards Self-supervised Occupancy Estimation via Differentiable Rendering and Occupancy Flow](https://mlsanity.com/paper/2402.12792/) (2024)

## Built on by (most cited first, 29 of 29)

- [FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving](https://mlsanity.com/paper/2505.17685/) (2025)
- [World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model](https://mlsanity.com/paper/2507.00603/) (2025)
- [A Comprehensive Survey on World Models for Embodied AI](https://mlsanity.com/paper/2510.16732/) (2025)
- [3D and 4D World Modeling: A Survey](https://mlsanity.com/paper/2509.07996/) (2025)
- [The Role of World Models in Shaping Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2502.10498/) (2025)
- [DriveX: Omni Scene Modeling for Learning Generalizable World Knowledge in Autonomous Driving](https://mlsanity.com/paper/2505.19239/) (2025)
- [WorldRFT: Latent World Model Planning with Reinforcement Fine-Tuning for Autonomous Driving](https://mlsanity.com/paper/2512.19133/) (2025)
- [WAM-Flow: Parallel Coarse-to-Fine Motion Planning via Discrete Flow Matching for Autonomous Driving](https://mlsanity.com/paper/2512.06112/) (2025)
- [SparseWorld: A Flexible, Adaptive, and Efficient 4D Occupancy World Model Powered by Sparse and Dynamic Queries](https://mlsanity.com/paper/2510.17482/) (2025)
- [GenieDrive: Towards Physics-Aware Driving World Model with 4D Occupancy Guided Video Generation](https://mlsanity.com/paper/2512.12751/) (2025)
- [Learning Vision-Language-Action World Models for Autonomous Driving](https://mlsanity.com/paper/2604.09059/) (2026)
- [UniUGP: Unifying Understanding, Generation, and Planing For End-to-end Autonomous Driving](https://mlsanity.com/paper/2512.09864/) (2025)
- [MindDriver: Introducing Progressive Multimodal Reasoning for Autonomous Driving](https://mlsanity.com/paper/2602.21952/) (2026)
- [Vega: Learning to Drive with Natural Language Instructions](https://mlsanity.com/paper/2603.25741/) (2026)
- [GASP: Unifying Geometric and Semantic Self-Supervised Pre-training for Autonomous Driving](https://mlsanity.com/paper/2503.15672/) (2025)
- [SparseWorld-TC: Trajectory-Conditioned Sparse Occupancy World Model](https://mlsanity.com/paper/2511.22039/) (2025)
- [SparseOccVLA: Bridging Occupancy and Vision-Language Models via Sparse Queries for Unified 4D Scene Understanding and Planning](https://mlsanity.com/paper/2601.06474/) (2026)
- [Enhancing Indoor Occupancy Prediction via Sparse Query-Based Multi-Level Consistent Knowledge Distillation](https://mlsanity.com/paper/2602.02318/) (2026)
- [SAMoE-VLA: A Scene Adaptive Mixture-of-Experts Vision-Language-Action Model for Autonomous Driving](https://mlsanity.com/paper/2603.08113/) (2026)
- [EvoDriveVLA: Evolving Driving VLA Models via Collaborative Perception-Planning Distillation](https://mlsanity.com/paper/2603.09465/) (2026)
- [Easy3D-Labels: Supervising Semantic Occupancy Estimation with 3D Pseudo-Labels for Automotive Perception](https://mlsanity.com/paper/2509.26087/) (2025)
- [The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Models](https://mlsanity.com/paper/2601.08876/) (2026)
- [DLWM: Dual Latent World Models enable Holistic Gaussian-centric Pre-training in Autonomous Driving](https://mlsanity.com/paper/2604.00969/) (2026)
- [Class-Distribution Guided Active Learning for 3D Occupancy Prediction in Autonomous Driving](https://mlsanity.com/paper/2603.27294/) (2026)
- [OccStress: Stress-Testing the 4D Occupancy Forecasting Chain](https://mlsanity.com/paper/2512.15621/) (2025)
- [ProOOD: Prototype-Guided Out-of-Distribution 3D Occupancy Prediction](https://mlsanity.com/paper/2604.01081/) (2026)
- [GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planning](https://mlsanity.com/paper/2605.17682/) (2026)
- [LiAuto-GeoX: Efficient Grounded Driving Transformer](https://mlsanity.com/paper/2606.05774/) (2026)
- [Geometry-Aware Spatio-Temporal Context Modeling for 4D Occupancy Forecasting](https://mlsanity.com/paper/2608.15279/) (2026)

## Abstract (arXiv)

Understanding world dynamics is crucial for planning in autonomous driving. Recent methods attempt to achieve this by learning a 3D occupancy world model that forecasts future surrounding scenes based on current observation. However, 3D occupancy labels are still required to produce promising results. Considering the high annotation cost for 3D outdoor scenes, we propose a semi-supervised vision-centric 3D occupancy world model, PreWorld, to leverage the potential of 2D labels through a novel two-stage training paradigm: the self-supervised pre-training stage and the fully-supervised fine-tuning stage. Specifically, during the pre-training stage, we utilize an attribute projection head to generate different attribute fields of a scene (e.g., RGB, density, semantic), thus enabling temporal supervision from 2D labels via volume rendering techniques. Furthermore, we introduce a simple yet effective state-conditioned forecasting module to recursively forecast future occupancy and ego trajectory in a direct manner. Extensive experiments on the nuScenes dataset validate the effectiveness and scalability of our method, and demonstrate that PreWorld achieves competitive performance across 3D occupancy prediction, 4D occupancy forecasting and motion planning tasks.