mlsanityData to 7 Oct 2026

GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splatting

arXiv 2408.11447 ↗First posted 21 Aug 2024cs.CVcited by 50

Not on the map. Why?

Abstract (arXiv)

We introduce GaussianOcc, a systematic method that investigates the two usages of Gaussian splatting for fully self-supervised and efficient 3D occupancy estimation in surround views. First, traditional methods for self-supervised 3D occupancy estimation still require ground truth 6D poses from sensors during training. To address this limitation, we propose Gaussian Splatting for Projection (GSP) module to provide accurate scale information for fully self-supervised training from adjacent view projection. Additionally, existing methods rely on volume rendering for final 3D voxel representation learning using 2D signals (depth maps, semantic maps), which is both time-consuming and less effective. We propose Gaussian Splatting from Voxel space (GSV) to leverage the fast rendering properties of Gaussian splatting. As a result, the proposed GaussianOcc method enables fully self-supervised (no ground truth pose) 3D occupancy estimation in competitive performance with low computational cost (2.7 times faster in training and 5 times faster in rendering). The relevant code is available in https://github.com/GANWANSHUI/GaussianOcc.git.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 44 of its 63 references that are in the map. First posted on arXiv 21 Aug 2024.

Builds on

44 of 63 references.

References · 44

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  5. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  6. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
  7. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  8. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  9. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  10. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  11. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
  12. 2023DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenescited by 359
32 more
  1. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
  2. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  3. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 340
  4. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  5. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 310
  6. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
  7. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 218
  8. 2023Scene as Occupancycited by 208
  9. 2023A Simple Framework for Open-Vocabulary Segmentation and Detectioncited by 197
  10. 20244D-Rotor Gaussian Splatting: Towards Efficient Novel View Synthesis for Dynamic Scenescited by 185
  11. 2023GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesiscited by 170
  12. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 162
  13. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 155
  14. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  15. 2022Vision-Centric BEV Perception: A Surveycited by 126
  16. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 126
  17. 2024textitS3Gaussian: Self-Supervised Street Gaussians for Autonomous Drivingcited by 105
  18. 2023FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugincited by 101
  19. 20243D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunitiescited by 77
  20. 2022SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimationcited by 76
  21. 2023Behind the Scenes: Density Fields for Single View Reconstructioncited by 63
  22. 2022SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fieldscited by 49
  23. 2022V4d: voxel for 4d novel view synthesiscited by 42
  24. 2023R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Camerascited by 39
  25. 2024A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspectivecited by 37
  26. 2023TBP-Former: Learning Temporal Bird's-Eye-View Pyramid for Joint Perception and Prediction in Vision-Centric Autonomous Drivingcited by 30
  27. 2023S4C: Self-Supervised Semantic Scene Completion with Neural Fieldscited by 27
  28. 2023Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Reviewcited by 25
  29. 2024GaussianBeV: 3D Gaussian Representation meets Perception Models for BeV Segmentationcited by 20
  30. 2024Neural Radiance Field in Autonomous Driving: A Surveycited by 19
  31. 2024SD-MVS: Segmentation-Driven Deformation Multi-View Stereo with Spherical Refinement and EM optimizationcited by 18
  32. 2024Towards Scale-Aware Full Surround Monodepth with Transformerscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 23; 2026: 21.

50 papers.

Sort:

By area:

  1. 2025A Survey of World Models for Autonomous Drivingcited by 60
  2. 2024GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understandingcited by 46
  3. 2024RenderWorld: World Model with Self-Supervised 3D Labelcited by 44
  4. 2025GaussianFlowOcc: Sparse and Weakly Supervised Occupancy Estimation using Gaussian Splatting and Temporal Flowcited by 24
  5. 2024ALOcc: Adaptive Lifting-Based 3D Semantic Occupancy and Cost Volume-Based Flow Predictionscited by 19
  6. 2025GaussRender: Learning 3D Occupancy with Gaussian Renderingcited by 17
  7. 2024Deep Height Decoupling for Precise Vision-based 3D Occupancy Predictioncited by 16
  8. 2024VisionPAD: A Vision-Centric Pre-training Paradigm for Autonomous Drivingcited by 15
  9. 2025AutoOcc: Automatic Open-Ended Semantic Occupancy Annotation via Vision-Language Guided Gaussian Splattingcited by 14
  10. 2025GaussianFormer3D: Multi-Modal Gaussian-based Semantic Occupancy Prediction with 3D Deformable Attentioncited by 14
  11. 2025AGO: Adaptive Grounding for Open World 3D Occupancy Predictioncited by 9
  12. 2025Rethinking Temporal Fusion with a Unified Gradient Descent View for 3D Semantic Occupancy Predictioncited by 9
38 more
  1. 2025VoxelSplat: Dynamic Gaussian Splatting as an Effective Loss for Occupancy and Flow Predictioncited by 9
  2. 2024GSRender: Deduplicated Occupancy Prediction via Weakly Supervised 3D Gaussian Splattingcited by 6
  3. 2025OneOcc: Semantic Occupancy Prediction for Legged Robots with a Single Panoramic Cameracited by 6
  4. 2025ShelfOcc: Native 3D Supervision beyond LiDAR for Vision-Based Occupancy Estimationcited by 6
  5. 2026OccAny: Generalized Unconstrained Urban 3D Occupancycited by 6
  6. 2025Progressive Gaussian Transformer with Anisotropy-aware Sampling for Open Vocabulary Occupancy Predictioncited by 4
  7. 2025Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systemscited by 4
  8. 2025GaussianFusionOcc: A Seamless Sensor Fusion Approach for 3D Occupancy Prediction Using 3D Gaussianscited by 3
  9. 2025ShelfGaussian: Shelf-Supervised Open-Vocabulary Gaussian-based 3D Scene Understandingcited by 3
  10. 2026Latent Gaussian Splatting for 4D Panoptic Occupancy Trackingcited by 3
  11. 2025Test-Time 3D Occupancy Predictioncited by 2
  12. 2025MS-Occ: Multi-Stage LiDAR-Camera Fusion for 3D Semantic Occupancy Predictioncited by 2
  13. 2025See through the Dark: Learning Illumination-affined Representations for Nighttime Occupancy Predictioncited by 2
  14. 2025GraphGSOcc: Semantic-Geometric Graph Transformer with Dynamic-Static Decoupling for 3D Gaussian Splatting-based Occupancy Predictioncited by 2
  15. 2025Easy3D-Labels: Supervising Semantic Occupancy Estimation with 3D Pseudo-Labels for Automotive Perceptioncited by 2
  16. 2025QueryOcc: Query-based Self-Supervision for 3D Semantic Occupancycited by 2
  17. 2026Dr.Occ: Depth- and Region-Guided 3D Occupancy from Surround-View Cameras for Autonomous Drivingcited by 2
  18. 2026DLWM: Dual Latent World Models enable Holistic Gaussian-centric Pre-training in Autonomous Drivingcited by 2
  19. 2026FreeOcc: Training-Free Embodied Open-Vocabulary Occupancy Predictioncited by 2
  20. 2026Gau-Occ: Geometry-Completed Gaussians for Multi-Modal 3D Occupancy Predictioncited by 1
  21. 2026Class-Distribution Guided Active Learning for 3D Occupancy Prediction in Autonomous Drivingcited by 1
  22. 2026Bridging 3D Gaussians and Semantic Occupancy for Comprehensive Open-Vocabulary Scene Understanding from Unposed Imagescited by 1
  23. 2025Deflickering Vision-Based Occupancy Networks through Lightweight Spatio-Temporal Correlationcited by 0
  24. 2025SuperQuadricOcc: Real-Time Self-Supervised Semantic Occupancy Estimation with Superquadric Volume Renderingcited by 0
  25. 2025HyGE-Occ: Hybrid View-Transformation with 3D Gaussian and Edge Priors for 3D Panoptic Occupancy Predictioncited by 0
  26. 2026FreeOcc: Training-free Panoptic Occupancy Prediction via Foundation Modelscited by 0
  27. 2026Difix3D-W: Distractor-Free Few-Shot 3D Gaussian Splatting in the Wildcited by 0
  28. 2026GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planningcited by 0
  29. 2026Deformable Gaussian Occupancy: Decoupling Rigid and Nonrigid Motion with Factorized Distillationcited by 0
  30. 2026UnsOcc: 3D Semantic Occupancy Prediction in Unstructured Scene via Rendering Fusioncited by 0
  31. 2026FLM-Occ: Feed-forward Likelihood Maximization for Efficient Indoor Occupancy Predictioncited by 0
  32. 2026GaussianFusion: Unified 3D Gaussian Representation for Multi-Modal Fusion Perceptioncited by 0
  33. 2026Targeted Structure Completion for Sparse-View 3D Reconstruction in Autonomous Drivingcited by 0
  34. 2026SparseOcc++: Geometry-Aware Sparse Latent Representation for Semantic Occupancy Predictioncited by 0
  35. 2026OccAnyScene: Towards Unified Indoor-Outdoor 3D Occupancy Predictioncited by 0
  36. 2026LetOccVote: Learning Weakly Supervised 3D Occupancy through Consensuscited by 0
  37. 2026SplatLabel: Pseudo-Labelling through 4D Gaussian Splattingcited by 0
  38. 2026TriO: Tri-Modal Unsupervised Occupancy World Model for Anything Perceptioncited by 0