mlsanityData to 14 Oct 2026

ST-GS: Vision-Based 3D Semantic Occupancy Prediction with Spatial-Temporal Gaussian Splatting

arXiv 2509.16552 ↗First posted 20 Sep 2025cs.CV cs.ROcited by 4

Not on the map. Why?

Abstract (arXiv)

3D occupancy prediction is critical for comprehensive scene understanding in vision-centric autonomous driving. Recent advances have explored utilizing 3D semantic Gaussians to model occupancy while reducing computational overhead, but they remain constrained by insufficient multi-view spatial interaction and limited multi-frame temporal consistency. To overcome these issues, in this paper, we propose a novel Spatial-Temporal Gaussian Splatting (ST-GS) framework to enhance both spatial and temporal modeling in existing Gaussian-based pipelines. Specifically, we develop a guidance-informed spatial aggregation strategy within a dual-mode attention mechanism to strengthen spatial interaction in Gaussian representations. Furthermore, we introduce a geometry-aware temporal fusion scheme that effectively leverages historical context to improve temporal continuity in scene completion. Extensive experiments on the large-scale nuScenes occupancy prediction benchmark showcase that our proposed approach not only achieves state-of-the-art performance but also delivers markedly better temporal consistency compared to existing Gaussian-based methods.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 29 of its 34 references that are in the map. First posted on arXiv 20 Sep 2025.

Builds on

29 of 34 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  3. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  5. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  6. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  7. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
  8. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
  9. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
  10. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 316
  11. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 313
  12. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 293
17 more
  1. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 223
  2. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 221
  3. 2020Atlas: End-to-End 3D Scene Reconstruction from Posed Imagescited by 206
  4. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 165
  5. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 157
  6. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 128
  7. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 115
  8. 2023Symphonize 3D Semantic Scene Completion with Contextual Instance Queriescited by 90
  9. 2024GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Predictioncited by 86
  10. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Predictioncited by 67
  11. 2024Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstructioncited by 56
  12. 2024ViewFormer: Exploring Spatiotemporal Modeling for Multi-View 3D Occupancy Perception via View-Guided Transformerscited by 37
  13. 2024MVGamba: Unify 3D Content Generation as State Space Sequence Modelingcited by 23
  14. 2025SEPT: Standard-Definition Map Enhanced Scene Perception and Topology Reasoning for Autonomous Drivingcited by 9
  15. 2025Occupancy Learning with Spatiotemporal Memorycited by 9
  16. 2025GoIRL: Graph-Oriented Inverse Reinforcement Learning for Multimodal Trajectory Predictioncited by 7
  17. 2025Foresight in Motion: Reinforcing Trajectory Prediction with Reward Heuristicscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.