mlsanityData to 7 Oct 2026

Holistic 3D Scene Understanding from a Single Image with Implicit Representation

arXiv 2103.06422 ↗First posted 11 Mar 2021cs.CVcited by 94

Not on the map. Why?

Abstract (arXiv)

We present a new pipeline for holistic 3D scene understanding from a single image, which could predict object shapes, object poses, and scene layout. As it is a highly ill-posed problem, existing methods usually suffer from inaccurate estimation of both shapes and layout especially for the cluttered scene due to the heavy occlusion between objects. We propose to utilize the latest deep implicit representation to solve this challenge. We not only propose an image-based local structured implicit network to improve the object shape estimation, but also refine the 3D object pose and scene layout via a novel implicit scene graph neural network that exploits the implicit local object features. A novel physical violation loss is also proposed to avoid incorrect context between objects. Extensive experiments demonstrate that our method outperforms the state-of-the-art methods in terms of object shape, scene layout estimation, and 3D object detection.

Key facts. Cited by 94 papers in the map (data through 7 Oct 2026). Builds on 44 of its 61 references that are in the map. First posted on arXiv 11 Mar 2021.

Builds on

44 of 61 references.

References · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2017Graph Attention Networkscited by 8,854
  6. 2017Inductive Representation Learning on Large Graphscited by 6,284
  7. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  8. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  9. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  10. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  11. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  12. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
32 more
  1. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  2. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  3. 2020Convolutional Occupancy Networkscited by 760
  4. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  5. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  6. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  7. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  8. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  9. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  10. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 332
  11. 2017Relation Networks for Object Detectioncited by 305
  12. 2018Graph R-CNN for Scene Graph Generationcited by 296
  13. 2019Learning Shape Templates with Structured Implicit Functionscited by 242
  14. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  15. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 170
  16. 2017Learning Efficient Point Cloud Generation for Dense 3D Object Reconstructioncited by 133
  17. 2016Generative and Discriminative Voxel Modeling with Convolutional Neural Networkscited by 128
  18. 2019Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboidscited by 123
  19. 2019PQ-NET: A Generative Part Seq2Seq Network for 3D Shapescited by 118
  20. 2019SDM-NET: Deep Generative Network for Structured Deformable Meshcited by 113
  21. 2019Deep Mesh Reconstruction from Single RGB Images via Topology Modification Networkscited by 106
  22. 2020Cross-Scale Internal Graph Neural Network for Image Super-Resolutioncited by 94
  23. 2016IM2CADcited by 78
  24. 2019Holistic++ Scene Understanding: Single-view 3D Holistic Scene Parsing and Human Pose Estimation with Human-Object Interaction and Physical Commonsensecited by 71
  25. 2018RGCNN: Regularized Graph CNN for Point Cloud Segmentationcited by 70
  26. 2018Holistic 3D Scene Parsing and Reconstruction from a Single RGB Imagecited by 69
  27. 2018Cooperative Holistic Scene Understanding: Unifying 3D Object, Layout, and Camera Pose Estimationcited by 51
  28. 2020SceneCAD: Predicting Object Alignments and Layouts in RGB-D Scanscited by 44
  29. 2019GEOMetrics: Exploiting Geometric Structure for Graph-Encoded Objectscited by 27
  30. 2020DOPS: Learning to Detect 3D Objects and Predict their 3D Shapescited by 27
  31. 2016A Coarse-to-Fine Indoor Layout Estimation (CFILE) Methodcited by 15
  32. 2017SeeThrough: Finding Chairs in Heavily Occluded Indoor Scene Imagescited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 15; 2023: 25; 2024: 15; 2025: 16; 2026: 15.

94 papers.

Sort:

By area:

  1. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  2. 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 113
  3. 2022GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Votingcited by 94
  4. 2025Visual Spatial Tuningcited by 85
  5. 2022ShAPO: Implicit Representations for Multi-Object Shape, Appearance, and Pose Optimizationcited by 70
  6. 2022CenterSnap: Single-Shot Multi-Object 3D Shape Reconstruction and Categorical 6D Pose and Size Estimationcited by 69
  7. 2022DM-NeRF: 3D Scene Geometry Decomposition and Manipulation from 2D Imagescited by 69
  8. 2023CARTO: Category and Joint Agnostic Reconstruction of ARTiculated Objectscited by 64
  9. 2024Gen3DSR: Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single Viewcited by 59
  10. 2022Human-Aware Object Placement for Visual Environment Reconstructioncited by 56
  11. 2022SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fieldscited by 49
  12. 2023DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Imagecited by 48
38 more
  1. 2024PhyRecon: Physically Plausible Neural Scene Reconstructioncited by 44
  2. 2022Towards High-Fidelity Single-view Holistic Reconstruction of Indoor Scenescited by 43
  3. 2022RBP-Pose: Residual Bounding Box Projection for Category-Level Pose Estimationcited by 43
  4. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 41
  5. 2023IST-Net: Prior-free Category-level Pose Estimation with Implicit Space Transformationcited by 40
  6. 2023Incremental 3D Semantic Scene Graph Prediction from RGB Sequencescited by 39
  7. 2023Single-view 3D Scene Reconstruction with High-fidelity Shape and Texturecited by 37
  8. 2021DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimizationcited by 33
  9. 2023LODE: Locally Conditioned Eikonal Implicit Scene Completion from Sparse LiDARcited by 31
  10. 2025Decompositional Neural Scene Reconstruction with Generative Diffusion Priorcited by 30
  11. 2022Learning 3D Scene Priors with 2D Supervisioncited by 24
  12. 2025DepR: Depth Guided Single-view Scene Reconstruction with Instance-level Diffusioncited by 23
  13. 2021Cerberus Transformer: Joint Semantic, Affordance and Attribute Parsingcited by 22
  14. 2021Pose2Room: Understanding 3D Scenes from Human Activitiescited by 22
  15. 2022Point Scene Understanding via Disentangled Instance Mesh Reconstructioncited by 20
  16. 2024NeRF-MAE: Masked AutoEncoders for Self-Supervised 3D Representation Learning for Neural Radiance Fieldscited by 20
  17. 2023RICO: Regularizing the Unobservable for Indoor Compositional Reconstructioncited by 19
  18. 2023PanoContext-Former: Panoramic Total Scene Understanding with a Transformercited by 19
  19. 2021Implicit Neural Representation Learning for Hyperspectral Image Super-Resolutioncited by 17
  20. 2022RangeUDF: Semantic Surface Reconstruction from 3D Point Cloudscited by 16
  21. 2023Generating Visual Spatial Description via Holistic 3D Scene Understandingcited by 16
  22. 20233D Copy-Paste: Physically Plausible Object Insertion for Monocular 3D Detectioncited by 15
  23. 2023ImGeoNet: Image-induced Geometry-aware Voxel Representation for Multi-view 3D Object Detectioncited by 14
  24. 2025Detection Based Part-level Articulated Object Reconstruction from Single RGBD Imagecited by 13
  25. 2023RayDF: Neural Ray-surface Distance Fields with Multi-view Consistencycited by 12
  26. 2023U-RED: Unsupervised 3D Shape Retrieval and Deformation for Partial Point Cloudscited by 11
  27. 2021Semi-supervised Implicit Scene Completion from Sparse LiDARcited by 10
  28. 20253D-RE-GEN: 3D Reconstruction of Indoor Scenes with a Generative Frameworkcited by 10
  29. 20263D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Imagecited by 9
  30. 2024Coherent 3D Scene Diffusion From a Single RGB Imagecited by 8
  31. 2022Neural Rendering in a Room: Amodal 3D Understanding and Free-Viewpoint Rendering for the Closed Scene Composed of Pre-Captured Objectscited by 7
  32. 2023NU-MCC: Multiview Compressive Coding with Neighborhood Decoder and Repulsive UDFcited by 7
  33. 2023ShapeMatcher: Self-Supervised Joint Shape Canonicalization, Segmentation, Retrieval and Deformationcited by 7
  34. 2024Robust 3D Shape Reconstruction in Zero-Shot from a Single Image in the Wildcited by 7
  35. 2024Unsupervised Template-assisted Point Cloud Shape Correspondence Networkcited by 7
  36. 2024Reconstructing Animals and the Wildcited by 7
  37. 2025Digital Twin Generation from Visual Data: A Surveycited by 7
  38. 2021Online Adaptation for Implicit Object Tracking and Shape Reconstruction in the Wildcited by 6

50 most cited of 94 All 94 (JSON)