ST-GS: Vision-Based 3D Semantic Occupancy Prediction with Spatial-Temporal Gaussian Splatting
Not on the map. Why?
Abstract (arXiv)
3D occupancy prediction is critical for comprehensive scene understanding in vision-centric autonomous driving. Recent advances have explored utilizing 3D semantic Gaussians to model occupancy while reducing computational overhead, but they remain constrained by insufficient multi-view spatial interaction and limited multi-frame temporal consistency. To overcome these issues, in this paper, we propose a novel Spatial-Temporal Gaussian Splatting (ST-GS) framework to enhance both spatial and temporal modeling in existing Gaussian-based pipelines. Specifically, we develop a guidance-informed spatial aggregation strategy within a dual-mode attention mechanism to strengthen spatial interaction in Gaussian representations. Furthermore, we introduce a geometry-aware temporal fusion scheme that effectively leverages historical context to improve temporal continuity in scene completion. Extensive experiments on the large-scale nuScenes occupancy prediction benchmark showcase that our proposed approach not only achieves state-of-the-art performance but also delivers markedly better temporal consistency compared to existing Gaussian-based methods.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 29 of its 34 references that are in the map. First posted on arXiv 20 Sep 2025.
Builds on
29 of 34 references.
References · 29
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2016Feature Pyramid Networks for Object Detectioncited by 5,036
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
- 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
- 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
- 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
- 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
- 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 316
- 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 313
- 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 293
17 more
- 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 223
- 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 221
- 2020Atlas: End-to-End 3D Scene Reconstruction from Posed Imagescited by 206
- 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 165
- 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 157
- 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 128
- 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 115
- 2023Symphonize 3D Semantic Scene Completion with Contextual Instance Queriescited by 90
- 2024GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Predictioncited by 86
- 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Predictioncited by 67
- 2024Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstructioncited by 56
- 2024ViewFormer: Exploring Spatiotemporal Modeling for Multi-View 3D Occupancy Perception via View-Guided Transformerscited by 37
- 2024MVGamba: Unify 3D Content Generation as State Space Sequence Modelingcited by 23
- 2025SEPT: Standard-Definition Map Enhanced Scene Perception and Topology Reasoning for Autonomous Drivingcited by 9
- 2025Occupancy Learning with Spatiotemporal Memorycited by 9
- 2025GoIRL: Graph-Oriented Inverse Reinforcement Learning for Multimodal Trajectory Predictioncited by 7
- 2025Foresight in Motion: Reinforcing Trajectory Prediction with Reward Heuristicscited by 6
Built on by
Citing papers per year: 2025: 2; 2026: 2.
4 papers.
- 2025UniSplat: Unified Spatio-Temporal Fusion via 3D Latent Scaffolds for Dynamic Driving Scene Reconstructioncited by 10
- 2026VG3S: Visual Geometry Grounded Gaussian Splatting for Semantic Occupancy Predictioncited by 5
- 2025TGSFormer: Scalable Temporal Gaussian Splatting for Embodied Semantic Scene Completioncited by 2
- 2026Learning Adaptive Semantic Gaussian Allocation for 3D Occupancycited by 0
- 2026Learning Adaptive Semantic Gaussian Allocation for 3D Occupancy24 Jul 2026
- 2026VG3S: Visual Geometry Grounded Gaussian Splatting for Semantic Occupancy Prediction6 Mar 2026
- 2025TGSFormer: Scalable Temporal Gaussian Splatting for Embodied Semantic Scene Completion29 Nov 2025
- 2025UniSplat: Unified Spatio-Temporal Fusion via 3D Latent Scaffolds for Dynamic Driving Scene Reconstruction6 Nov 2025