Geometry-Aware Spatio-Temporal Context Modeling for 4D Occupancy Forecasting
Not on the map. Why?
Abstract (arXiv)
4D occupancy forecasting models the spatio-temporal evolution of 3D scenes and is crucial for autonomous driving, especially for corner-case simulation. Existing methods often rely on discrete tokenization followed by autoregressive prediction, yet struggle with geometric distortion in static structures and inconsistent temporal coherence over the forecasting horizon. In this work, we propose a Geometry-Aware Spatio-Temporal context modeling method (GAST) for 4D occupancy forecasting, built upon progressive explicit-implicit generation and dual-path spatio-temporal modeling. Specifically, the generation module produces per-frame occupancy with high geometric fidelity and semantic plausibility through pose-driven warping, motion-aware feature modulation, and attention-based feature refinement. Subsequently, the spatio-temporal module enhances spatial consistency through global context aggregation while capturing scene evolution through temporal dynamics extraction. This unified design enables joint optimization of historical reconstruction and future forecasting in an end-to-end manner. Extensive experiments on Occ3D-nuScenes demonstrate the superiority of our method, outperforming the state-of-the-art by 7.67% in mIoU and 6.44% in IoU with a 2.84x speedup, while maintaining strong performance in long-term forecasting.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 41 of its 48 references that are in the map. First posted on arXiv 15 Aug 2026.
Builds on
41 of 48 references.
References · 41
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2017Neural Discrete Representation Learningcited by 4,712
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
- 2020Center-based 3D Object Detection and Trackingcited by 1,144
- 2023HexPlane: A Fast Representation for Dynamic Scenescited by 587
- 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
- 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
- 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 316
29 more
- 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 313
- 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 233
- 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 223
- 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 221
- 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 166
- 20222DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Cloudscited by 158
- 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 106
- 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Drivingcited by 98
- 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Modelcited by 95
- 2024OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Drivingcited by 83
- 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 78
- 2024DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Modelcited by 61
- 2021EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentationcited by 59
- 2024Context and Geometry Aware Voxel Transformer for Semantic Scene Completioncited by 50
- 2024Co-Occ: Coupling Explicit Feature Fusion with Volume Rendering Regularization for Multi-Modal 3D Semantic Occupancy Predictioncited by 48
- 2024RenderWorld: World Model with Self-Supervised 3D Labelcited by 46
- 2024DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenescited by 38
- 2021Semantic Segmentation-assisted Scene Completion for LiDAR Point Cloudscited by 37
- 2024DrivingSphere: Building a High-fidelity 4D World for Closed-loop Simulationcited by 34
- 2023SSC-RS: Elevate LiDAR Semantic Scene Completion with Representation Separation and BEV Fusioncited by 30
- 2024OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Drivingcited by 29
- 2025Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Drivingcited by 29
- 2025Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Modelscited by 28
- 2025STCOcc: Sparse Spatial-Temporal Cascade Renovation for 3D Occupancy and Scene Flow Predictioncited by 23
- 2024An Efficient Occupancy World Model via Decoupled Dynamic Flow and Image-assisted Trainingcited by 18
- 2025COME: Adding Scene-Centric Forecasting Control to Occupancy World Modelcited by 14
- 2025OccTENS: 3D Occupancy World Model via Temporal Next-Scale Predictioncited by 9
- 2024Robust 3D Semantic Occupancy Prediction with Calibration-free Spatial Transformationcited by 7
- 2025VisHall3D: Monocular Semantic Scene Completion from Reconstructing the Visible Regions to Hallucinating the Invisible Regionscited by 7
Built on by
Not cited yet.