mlsanityData to 7 Oct 2026

MonoScene: Monocular 3D Semantic Scene Completion

arXiv 2112.00726 ↗First posted 1 Dec 2021cs.CV cs.AI cs.ROcited by 357

Not on the map. Why?

Cited by 25 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

MonoScene proposes a 3D Semantic Scene Completion (SSC) framework, where the dense geometry and semantics of a scene are inferred from a single monocular RGB image. Different from the SSC literature, relying on 2.5 or 3D input, we solve the complex problem of 2D to 3D scene reconstruction while jointly inferring its semantics. Our framework relies on successive 2D and 3D UNets bridged by a novel 2D-3D features projection inspiring from optics and introduces a 3D context relation prior to enforce spatio-semantic consistency. Along with architectural contributions, we introduce novel global scene and local frustums losses. Experiments show we outperform the literature on all metrics and datasets while hallucinating plausible scenery even beyond the camera field of view. Our code and trained models are available at https://github.com/cv-rits/MonoScene.

Key facts. Cited by 357 papers in the map (data through 7 Oct 2026). Builds on 63 of its 83 references that are in the map. First posted on arXiv 1 Dec 2021.

Builds on

63 of 83 references.

References · 63

  1. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  4. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  6. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  7. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
  8. 2018Dual Attention Network for Scene Segmentationcited by 1,001
  9. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  10. 2020Convolutional Occupancy Networkscited by 760
  11. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
  12. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
38 more
  1. 2020AdaBins: Depth Estimation using Adaptive Binscited by 596
  2. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  3. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  4. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  5. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  6. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  7. 20163DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructionscited by 333
  8. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 332
  9. 2018Context Encoding for Semantic Segmentationcited by 289
  10. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 222
  11. 2019BSP-Net: Generating Compact Meshes via Binary Space Partitioningcited by 205
  12. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 187
  13. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  14. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 170
  15. 2019Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Imagescited by 164
  16. 2020Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wildcited by 152
  17. 2018Scan2CAD: Learning CAD Model Alignment in RGB-D Scanscited by 142
  18. 2018Improving Semantic Segmentation via Video Propagation and Label Relaxationcited by 134
  19. 2020Anisotropic Convolutional Networks for 3D Semantic Scene Completioncited by 126
  20. 20203D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Priorcited by 124
  21. 2020Context Prior for Scene Segmentationcited by 121
  22. 2018Learning a Discriminative Feature Network for Semantic Segmentationcited by 114
  23. 2019HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentationcited by 107
  24. 2020S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Cloudscited by 102
  25. 2018LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Imagecited by 94
  26. 2021Holistic 3D Scene Understanding from a Single Image with Implicit Representationcited by 94
  27. 2019Image-based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Eracited by 87
  28. 2020Pix2Vox++: Multi-scale Context-aware 3D Object Reconstruction from Single and Multiple Imagescited by 79
  29. 2016IM2CADcited by 78
  30. 2019SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scanscited by 77
  31. 2020Semantic Scene Completion using Local Deep Implicit Functions on LiDAR Datacited by 74
  32. 20213D Semantic Scene Completion: a Surveycited by 74
  33. 2021Panoptic 3D Scene Reconstruction From a Single RGB Imagecited by 70
  34. 2018Holistic 3D Scene Parsing and Reconstruction from a Single RGB Imagecited by 69
  35. 2019Efficient Semantic Scene Completion Network with Spatial Group Convolutioncited by 66
  36. 2019RGBD Based Dimensional Decomposition Residual Network for 3D Semantic Scene Completioncited by 54
  37. 2021Semantic Scene Completion via Integrating Instances and Scene in-the-Loopcited by 54
  38. 2018Two Stream 3D Semantic Scene Completioncited by 48

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 58; 2024: 104; 2025: 115; 2026: 76.

357 papers.

Sort:

By area:

  1. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
  2. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  3. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 310
  4. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 306
  5. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
  6. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 230
  7. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 218
  8. 2023Scene as Occupancycited by 208
  9. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 162
  10. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 155
  11. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 146
  12. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
38 more
  1. 2022Vision-Centric BEV Perception: A Surveycited by 126
  2. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 126
  3. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 114
  4. 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 102
  5. 2023FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugincited by 101
  6. 2023COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Predictioncited by 94
  7. 2024SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Predictioncited by 94
  8. 2023Symphonize 3D Semantic Scene Completion with Contextual Instance Queriescited by 88
  9. 2023NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Spacecited by 88
  10. 2023OccDepth: A Depth-Aware Method for 3D Semantic Scene Completioncited by 85
  11. 2024GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Predictioncited by 83
  12. 2023SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Drivingcited by 72
  13. 2023Multiview Compressive Coding for 3D Reconstructioncited by 71
  14. 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Drivingcited by 67
  15. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Predictioncited by 66
  16. 20253D and 4D World Modeling: A Surveycited by 64
  17. 2023Fully Sparse 3D Occupancy Predictioncited by 61
  18. 2025A Survey of World Models for Autonomous Drivingcited by 60
  19. 2024OPUS: Occupancy Prediction Using a Sparse Setcited by 56
  20. 2024SemCity: Semantic Scene Generation with Triplane Diffusioncited by 53
  21. 2024POP-3D: Open-Vocabulary 3D Occupancy Prediction from Imagescited by 52
  22. 2023PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Predictioncited by 51
  23. 2024FastOcc: Accelerating 3D Occupancy Prediction by Fusing the 2D Bird's-Eye View and Perspective Viewcited by 51
  24. 2023Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applicationscited by 50
  25. 2024Context and Geometry Aware Voxel Transformer for Semantic Scene Completioncited by 50
  26. 2024GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splattingcited by 50
  27. 2022SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fieldscited by 49
  28. 2023OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queriescited by 48
  29. 2024Co-Occ: Coupling Explicit Feature Fusion with Volume Rendering Regularization for Multi-Modal 3D Semantic Occupancy Predictioncited by 46
  30. 2024GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understandingcited by 46
  31. 2024RenderWorld: World Model with Self-Supervised 3D Labelcited by 44
  32. 2024MonoOcc: Digging into Monocular Semantic Occupancy Predictioncited by 43
  33. 2024Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillationcited by 43
  34. 2024EmbodiedOcc: Embodied 3D Occupancy Prediction for Vision-based Online Scene Understandingcited by 43
  35. 2023PaSCo: Urban 3D Panoptic Scene Completion with Uncertainty Awarenesscited by 42
  36. 2023Camera-based 3D Semantic Scene Completion with Sparse Guidance Networkcited by 40
  37. 2024Gaussian Splatting: 3D Reconstruction and Novel View Synthesis, a Reviewcited by 40
  38. 2023OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environmentscited by 39

50 most cited of 357 All 357 (JSON)