mlsanityData to 7 Oct 2026

OccDepth: A Depth-Aware Method for 3D Semantic Scene Completion

arXiv 2302.13540 ↗First posted 27 Feb 2023cs.CVcited by 85

Not on the map. Why?

Abstract (arXiv)

3D Semantic Scene Completion (SSC) can provide dense geometric and semantic scene representations, which can be applied in the field of autonomous driving and robotic systems. It is challenging to estimate the complete geometry and semantics of a scene solely from visual images, and accurate depth information is crucial for restoring 3D geometry. In this paper, we propose the first stereo SSC method named OccDepth, which fully exploits implicit depth information from stereo images (or RGBD images) to help the recovery of 3D geometric structures. The Stereo Soft Feature Assignment (Stereo-SFA) module is proposed to better fuse 3D depth-aware features by implicitly learning the correlation between stereo images. In particular, when the input are RGBD image, a virtual stereo images can be generated through original RGB image and depth map. Besides, the Occupancy Aware Depth (OAD) module is used to obtain geometry-aware 3D features by knowledge distillation using pre-trained depth models. In addition, a reformed TartanAir benchmark, named SemanticTartanAir, is provided in this paper for further testing our OccDepth method on SSC task. Compared with the state-of-the-art RGB-inferred SSC method, extensive experiments on SemanticKITTI show that our OccDepth method achieves superior performance with improving +4.82% mIoU, of which +2.49% mIoU comes from stereo images and +2.33% mIoU comes from our proposed depth-aware method. Our code and trained models are available at https://github.com/megvii-research/OccDepth.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). Builds on 34 of its 54 references that are in the map. First posted on arXiv 27 Feb 2023.

Builds on

34 of 54 references.

References · 34

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  3. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  4. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  5. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  6. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  7. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  8. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  9. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  10. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  11. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 326
  12. 2019Structured Knowledge Distillation for Dense Predictioncited by 277
22 more
  1. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 222
  2. 2022Unifying Voxel-based Representation with Transformer for 3D Object Detectioncited by 202
  3. 2020S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Cloudscited by 102
  4. 2019SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scanscited by 77
  5. 2020Semantic Scene Completion using Local Deep Implicit Functions on LiDAR Datacited by 74
  6. 20213D Semantic Scene Completion: a Surveycited by 74
  7. 2021Panoptic 3D Scene Reconstruction From a Single RGB Imagecited by 70
  8. 2022MonoDistill: Learning Spatial Features for Monocular 3D Object Detectioncited by 68
  9. 2020Inter-Region Affinity Distillation for Road Marking Segmentationcited by 58
  10. 2019RGBD Based Dimensional Decomposition Residual Network for 3D Semantic Scene Completioncited by 54
  11. 2022SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fieldscited by 49
  12. 2018Two Stream 3D Semantic Scene Completioncited by 48
  13. 2019Refine and Distill: Exploiting Cycle-Inconsistency and Knowledge Distillation for Unsupervised Monocular Depth Estimationcited by 48
  14. 2021Semantic Segmentation-assisted Scene Completion for LiDAR Point Cloudscited by 37
  15. 2019Student Becoming the Master: Knowledge Amalgamation for Joint Scene Parsing, Depth Estimation, and Morecited by 29
  16. 2020Attention-based Multi-modal Fusion Network for Semantic Scene Completioncited by 28
  17. 2020Center3D: Center-based Monocular 3D Object Detection with Joint Depth Understandingcited by 25
  18. 2020SCFusion: Real-time Incremental Scene Reconstruction with Semantic Completioncited by 25
  19. 2021Not All Voxels Are Equal: Semantic Scene Completion from the Point-Voxel Perspectivecited by 22
  20. 2022Boosting 3D Object Detection by Simulating Multimodality on Point Cloudscited by 22
  21. 2021Voxel-based 3D Detection and Reconstruction of Multiple Objects from a Single Imagecited by 20
  22. 2021Joint stereo 3D object detection and implicit surface reconstructioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 22; 2024: 30; 2025: 26; 2026: 7.

85 papers.

Sort:

By area:

  1. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  2. 2023Scene as Occupancycited by 208
  3. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 155
  4. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 126
  5. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 114
  6. 2023COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Predictioncited by 94
  7. 2023Symphonize 3D Semantic Scene Completion with Contextual Instance Queriescited by 88
  8. 2024GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Predictioncited by 83
  9. 2024SemCity: Semantic Scene Generation with Triplane Diffusioncited by 53
  10. 2024Context and Geometry Aware Voxel Transformer for Semantic Scene Completioncited by 50
  11. 2023OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queriescited by 48
  12. 2023Camera-based 3D Semantic Scene Completion with Sparse Guidance Networkcited by 40
38 more
  1. 2024A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspectivecited by 37
  2. 2024Occupancy as Set of Pointscited by 37
  3. 2024ViewFormer: Exploring Spatiotemporal Modeling for Multi-View 3D Occupancy Perception via View-Guided Transformerscited by 36
  4. 2023Learning Occupancy for Monocular 3D Object Detectioncited by 35
  5. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
  6. 2024Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehiclescited by 33
  7. 2023UniWorld: Autonomous Driving Pre-training via World Modelscited by 32
  8. 2023DepthSSC: Monocular 3D Semantic Scene Completion via Depth-Spatial Alignment and Voxel Adaptationcited by 30
  9. 2024Vision-based 3D occupancy prediction in autonomous driving: a review and outlookcited by 29
  10. 2023S4C: Self-Supervised Semantic Scene Completion with Neural Fieldscited by 27
  11. 2023Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Reviewcited by 25
  12. 2023OVO: Open-Vocabulary Occupancycited by 24
  13. 2024Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunitiescited by 24
  14. 2024The RoboDrive Challenge: Drive Anytime Anywhere in Any Conditioncited by 23
  15. 2024CVT-Occ: Cost Volume Temporal Fusion for 3D Occupancy Predictioncited by 23
  16. 2024OccFlowNet: Towards Self-supervised Occupancy Estimation via Differentiable Rendering and Occupancy Flowcited by 22
  17. 2024VEON: Vocabulary-Enhanced Occupancy Predictioncited by 22
  18. 2025STCOcc: Sparse Spatial-Temporal Cascade Renovation for 3D Occupancy and Scene Flow Predictioncited by 20
  19. 2024Deep Height Decoupling for Precise Vision-based 3D Occupancy Predictioncited by 16
  20. 2024GAFusion: Adaptive Fusing LiDAR and Camera with Multiple Guidance for 3D Object Detectioncited by 14
  21. 2025GaussianFormer3D: Multi-Modal Gaussian-based Semantic Occupancy Prediction with 3D Deformable Attentioncited by 14
  22. 2023One at a Time: Progressive Multi-step Volumetric Probability Learning for Reliable 3D Scene Perceptioncited by 13
  23. 2024OccFusion: Depth Estimation Free Multi-sensor Fusion for 3D Occupancy Predictioncited by 13
  24. 2024Offboard Occupancy Refinement with Hybrid Propagation for Autonomous Drivingcited by 12
  25. 2024Know Your Neighbors: Improving Single-View Reconstruction via Spatial Vision-Language Reasoningcited by 12
  26. 2025Feed-Forward SceneDINO for Unsupervised Semantic Scene Completioncited by 11
  27. 2024SurroundSDF: Implicit 3D Scene Understanding Based on Signed Distance Fieldcited by 10
  28. 2023SOGDet: Semantic-Occupancy Guided Multi-view 3D Object Detectioncited by 9
  29. 2024PanoSSC: Exploring Monocular Panoptic 3D Scene Reconstruction for Autonomous Drivingcited by 9
  30. 2025VoxelSplat: Dynamic Gaussian Splatting as an Effective Loss for Occupancy and Flow Predictioncited by 9
  31. 2024OccLoff: Learning Optimized Feature Fusion for 3D Occupancy Predictioncited by 7
  32. 2025LaRI: Layered Ray Intersections for Single-view 3D Geometric Reasoningcited by 7
  33. 2025VisHall3D: Monocular Semantic Scene Completion from Reconstructing the Visible Regions to Hallucinating the Invisible Regionscited by 7
  34. 2025SplatSSC: Decoupled Depth-Guided Gaussian Splatting for Semantic Scene Completioncited by 7
  35. 2024OccTransformer: Improving BEVFormer for 3D camera-only occupancy predictioncited by 6
  36. 2024ZOPP: A Framework of Zero-shot Offboard Panoptic Perception for Autonomous Drivingcited by 6
  37. 2025Disentangling Instance and Scene Contexts for 3D Semantic Scene Completioncited by 6
  38. 2023Occ2Net: Robust Image Matching Based on 3D Occupancy Estimation for Occluded Regionscited by 5

50 most cited of 85 All 85 (JSON)