mlsanityData to 7 Oct 2026

Semantic Scene Completion from a Single Depth Image

arXiv 1611.08974 ↗First posted 28 Nov 2016cs.CVcited by 468

Not on the map. Why?

Cited by 24 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper focuses on semantic scene completion, a task for producing a complete 3D voxel representation of volumetric occupancy and semantic labels for a scene from a single-view depth map observation. Previous work has considered scene completion and semantic labeling of depth maps separately. However, we observe that these two problems are tightly intertwined. To leverage the coupled nature of these two tasks, we introduce the semantic scene completion network (SSCNet), an end-to-end 3D convolutional network that takes a single depth image as input and simultaneously outputs occupancy and semantic labels for all voxels in the camera view frustum. Our network uses a dilation-based 3D context module to efficiently expand the receptive field and enable 3D context learning. To train our network, we construct SUNCG - a manually created large-scale dataset of synthetic 3D scenes with dense volumetric annotations. Our experiments demonstrate that the joint model outperforms methods addressing each task in isolation and outperforms alternative approaches on the semantic scene completion task.

Key facts. Cited by 468 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 2; 2019: 1; 2020: 0; 2021: 89; 2022: 70; 2023: 83; 2024: 95; 2025: 80; 2026: 48.

468 papers.

Sort:

By area:

  1. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  2. 2023Point Transformer V3: Simpler, Faster, Strongercited by 579
  3. 2022Point Transformer V2: Grouped Vector Attention and Partition-based Poolingcited by 384
  4. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  5. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  6. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  7. 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
  8. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 310
  9. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 306
  10. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  11. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
  12. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
38 more
  1. 2023SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesiscited by 247
  2. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 218
  3. 2023Scene as Occupancycited by 208
  4. 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
  5. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  6. 20222DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Cloudscited by 157
  7. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 155
  8. 2023Infinite Photorealistic Worlds using Procedural Generationcited by 142
  9. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  10. 20214DComplete: Non-Rigid Motion Estimation Beyond the Observable Surfacecited by 123
  11. 2021NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenescited by 119
  12. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  13. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 114
  14. 2022Map-free Visual Relocalization: Metric Pose Relative to a Single Imagecited by 112
  15. 2022Habitat-Matterport 3D Semantics Datasetcited by 109
  16. 2021Bidirectional Projection Network for Cross Dimension Scene Understandingcited by 95
  17. 2022Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Modelscited by 95
  18. 2022PanoFormer: Panorama Transformer for Indoor 360 Depth Estimationcited by 94
  19. 2023COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Predictioncited by 94
  20. 2024SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Predictioncited by 94
  21. 2021UniFuse: Unidirectional Fusion for 360circ Panorama Depth Estimationcited by 91
  22. 2021Variational Transformer Networks for Layout Generationcited by 88
  23. 2023Symphonize 3D Semantic Scene Completion with Contextual Instance Queriescited by 88
  24. 2023NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Spacecited by 88
  25. 2021Guided Point Contrastive Learning for Semi-supervised Point Cloud Semantic Segmentationcited by 87
  26. 2024I-Design: Personalized LLM Interior Designercited by 87
  27. 2023OccDepth: A Depth-Aware Method for 3D Semantic Scene Completioncited by 85
  28. 2023Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Trainingcited by 85
  29. 2024GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Predictioncited by 83
  30. 2024Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localizationcited by 81
  31. 2023SCPNet: Semantic Scene Completion on Point Cloudcited by 80
  32. 2022Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracingcited by 78
  33. 2023UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generationcited by 76
  34. 20213D Semantic Scene Completion: a Surveycited by 74
  35. 2023SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Drivingcited by 72
  36. 2023Multiview Compressive Coding for 3D Reconstructioncited by 71
  37. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 71
  38. 2021Panoptic 3D Scene Reconstruction From a Single RGB Imagecited by 70

50 most cited of 468 All 468 (JSON)