mlsanityData to 13 Oct 2026

Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene

arXiv 1712.01812 ↗First posted 5 Dec 2017cs.CVcited by 50

Not on the map. Why?

Abstract (arXiv)

The goal of this paper is to take a single 2D image of a scene and recover the 3D structure in terms of a small set of factors: a layout representing the enclosing surfaces as well as a set of objects represented in terms of shape and pose. We propose a convolutional neural network-based approach to predict this representation and benchmark it on a large dataset of indoor scenes. Our experiments evaluate a number of practical design questions, demonstrate that we can infer this representation, and quantitatively and qualitatively demonstrate its merits compared to alternate representations.

Key facts. Cited by 50 papers in the map (data through 13 Oct 2026). First posted on arXiv 5 Dec 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 16; 2022: 13; 2023: 5; 2024: 12; 2025: 3; 2026: 1.

50 papers.

Sort:

By area:

  1. 2022Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wildcited by 159
  2. 2021Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphscited by 83
  3. 2022Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applicationscited by 78
  4. 2025RayZer: A Self-supervised Large View Synthesis Modelcited by 75
  5. 2022DM-NeRF: 3D Scene Geometry Decomposition and Manipulation from 2D Imagescited by 69
  6. 2024Gen3DSR: Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single Viewcited by 60
  7. 2023CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusioncited by 54
  8. 2022Few-View Object Reconstruction with Unknown Categories and Camera Posescited by 53
  9. 2022DiffDreamer: Towards Consistent Unsupervised Single-view Scene Extrapolation with Conditional Diffusion Modelscited by 51
  10. 2022Towards High-Fidelity Single-view Holistic Reconstruction of Indoor Scenescited by 44
  11. 2021Planar Surface Reconstruction from Sparse Viewscited by 40
  12. 2021D3D-HOI: Dynamic 3D Human-Object Interactions from Videoscited by 39
38 more
  1. 2021SAIL-VOS 3D: A Synthetic Dataset and Baselines for Object Detection and 3D Mesh Reconstruction from Video Datacited by 38
  2. 2022Understanding 3D Object Articulation in Internet Videoscited by 36
  3. 2024Dynamic 3D Gaussian Fields for Urban Areascited by 35
  4. 2022Learning to Imitate Object Interactions from Internet Videoscited by 32
  5. 2021Indoor Scene Generation from a Collection of Semantic-Segmented Depth Imagescited by 27
  6. 2024Multi-Level Neural Scene Graphs for Dynamic Urban Environmentscited by 27
  7. 20233DiffTection: 3D Object Detection with Geometry-Aware Diffusion Featurescited by 26
  8. 2022Learning 3D Scene Priors with 2D Supervisioncited by 24
  9. 2021Monte Carlo Scene Search for 3D Scene Understandingcited by 19
  10. 2021CoCoNets: Continuous Contrastive 3D Scene Representationscited by 19
  11. 2022Learning 3D Object Shape and Layout without 3D Supervisioncited by 19
  12. 2023PanoContext-Former: Panoramic Total Scene Understanding with a Transformercited by 19
  13. 2021Structure from Silence: Learning Scene Structure from Ambient Soundcited by 17
  14. 2022RayTran: 3D pose estimation and shape reconstruction of multiple objects from videos with ray-traced transformerscited by 17
  15. 2022RangeUDF: Semantic Surface Reconstruction from 3D Point Cloudscited by 16
  16. 2024Gaussian Splatting in Stylecited by 16
  17. 2022im2nerf: Image to Neural Radiance Field in the Wildcited by 11
  18. 2024External Knowledge Enhanced 3D Scene Generation from Sketchcited by 9
  19. 2024Coherent 3D Scene Diffusion From a Single RGB Imagecited by 8
  20. 2021What's Behind the Couch? Directed Ray Distance Functions (DRDF) for 3D Scene Reconstructioncited by 7
  21. 2024Lay-A-Scene: Personalized 3D Object Arrangement Using Text-to-Image Priorscited by 7
  22. 20243DFIRES: Few Image 3D REconstruction for Scenes with Hidden Surfacecited by 6
  23. 2026REST3D: Reconstructing Physically Stable 3D Scenes from a Single Imagecited by 6
  24. 2021Recognizing Scenes from Novel Viewpointscited by 5
  25. 2024V-MIND: Building Versatile Monocular Indoor 3D Detector with Diverse 2D Annotationscited by 4
  26. 2021Towards Panoptic 3D Parsing for Single Image in the Wildcited by 3
  27. 2024TreeSBA: Tree-Transformer for Self-Supervised Sequential Brick Assemblycited by 3
  28. 2021Equine Pain Behavior Classification via Self-Supervised Disentangled Pose Representationcited by 2
  29. 2024CatFree3D: Category-agnostic 3D Object Detection with Diffusioncited by 2
  30. 2025Measuring Physical Plausibility of 3D Human Poses Using Physics Simulationcited by 2
  31. 2025DreamAnywhere: Object-Centric Panoramic 3D Scene Generationcited by 2
  32. 2021Monocular Multi-Layer Layout Estimation for Warehouse Rackscited by 1
  33. 2021AutoLay: Benchmarking amodal layout estimation for autonomous drivingcited by 1
  34. 2021Joint stereo 3D object detection and implicit surface reconstructioncited by 1
  35. 2024LAM3D: Leveraging Attention for Monocular 3D Object Detectioncited by 1
  36. 2021Fusing RGBD Tracking and Segmentation Tree Sampling for Multi-Hypothesis Volumetric Segmentationcited by 0
  37. 2023Explicit3D: Graph Network with Spatial Inference for Single Image 3D Object Detectioncited by 0
  38. 2023Depth self-supervision for single image novel view synthesiscited by 0