mlsanityData to 7 Oct 2026

Voxel-based 3D Detection and Reconstruction of Multiple Objects from a Single Image

arXiv 2111.03098 ↗First posted 4 Nov 2021cs.CVcited by 20

Not on the map. Why?

Abstract (arXiv)

Inferring 3D locations and shapes of multiple objects from a single 2D image is a long-standing objective of computer vision. Most of the existing works either predict one of these 3D properties or focus on solving both for a single object. One fundamental challenge lies in how to learn an effective representation of the image that is well-suited for 3D detection and reconstruction. In this work, we propose to learn a regular grid of 3D voxel features from the input image which is aligned with 3D scene space via a 3D feature lifting operator. Based on the 3D voxel features, our novel CenterNet-3D detection head formulates the 3D detection as keypoint detection in the 3D space. Moreover, we devise an efficient coarse-to-fine reconstruction module, including coarse-level voxelization and a novel local PCA-SDF shape representation, which enables fine detail reconstruction and one order of magnitude faster inference than prior methods. With complementary supervision from both 3D detection and reconstruction, one enables the 3D voxel features to be geometry and context preserving, benefiting both tasks.The effectiveness of our approach is demonstrated through 3D detection and reconstruction in single object and multiple object scenarios.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 57 of its 77 references that are in the map. First posted on arXiv 4 Nov 2021.

Builds on

57 of 77 references.

References · 57

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  6. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  7. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
  8. 2019Objects as Pointscited by 1,172
  9. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  10. 2020Neural Sparse Voxel Fieldscited by 959
  11. 2017Focal Loss for Dense Object Detectioncited by 749
  12. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
38 more
  1. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  2. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  3. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  4. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  5. 2020Local Implicit Grid Representations for 3D Scenescited by 372
  6. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 326
  7. 2020Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstructioncited by 321
  8. 2018Generating 3D faces using Convolutional Mesh Autoencoderscited by 267
  9. 2019M3D-RPN: Monocular 3D Region Proposal Network for Object Detectioncited by 251
  10. 2019Learning Shape Templates with Structured Implicit Functionscited by 242
  11. 2016Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesiscited by 219
  12. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  13. 2019Disentangling Monocular 3D Object Detectioncited by 215
  14. 2019BSP-Net: Generating Compact Meshes via Binary Space Partitioningcited by 205
  15. 2020SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimationcited by 202
  16. 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 201
  17. 2019CvxNet: Learnable Convex Decompositioncited by 173
  18. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 170
  19. 2020MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationshipscited by 153
  20. 2017CNN-SLAM: Real-time dense monocular SLAM with learned depth predictioncited by 152
  21. 2018Scan2CAD: Learning CAD Model Alignment in RGB-D Scanscited by 142
  22. 2018Visual Object Networks: Image Generation with Disentangled 3D Representationcited by 124
  23. 2020Kinematic 3D Object Detection in Monocular Videocited by 116
  24. 2017MarrNet: 3D Shape Reconstruction via 2.5D Sketchescited by 108
  25. 2019Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformationcited by 103
  26. 2017Deep MANTA: A Coarse-to-fine Many-Task Network for joint 2D and 3D vehicle analysis from monocular imagecited by 95
  27. 2015Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Viewscited by 94
  28. 2018Holistic 3D Scene Parsing and Reconstruction from a Single RGB Imagecited by 69
  29. 2019Deep Fitting Degree Scoring Network for Monocular 3D Object Detectioncited by 67
  30. 2021GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detectioncited by 66
  31. 2014Viewpoints and Keypointscited by 59
  32. 2018Cooperative Holistic Scene Understanding: Unifying 3D Object, Layout, and Camera Pose Estimationcited by 51
  33. 2020FroDO: From Detections to 3D Objectscited by 51
  34. 2019Disentangled Representation Learning for 3D Face Shapecited by 48
  35. 2020CoReNet: Coherent 3D scene reconstruction from a single RGB imagecited by 45
  36. 2020From Points to Multi-Object 3D Reconstructioncited by 29
  37. 2020Learning Implicit Functions for Topology-Varying Dense 3D Shape Correspondencecited by 27
  38. 20193D Face Modeling From Diverse Raw Scan Datacited by 25

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 5; 2023: 5; 2024: 5; 2025: 3.

20 papers.

Sort:

  1. 2023OccDepth: A Depth-Aware Method for 3D Semantic Scene Completioncited by 85
  2. 2024RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detectioncited by 82
  3. 2021ROCA: Robust CAD Model Retrieval and Alignment from a Single Imagecited by 66
  4. 2022Faster VoxelPose: Real-time 3D Human Pose Estimation by Orthographic Projectioncited by 39
  5. 2023Tame a Wild Camera: In-the-Wild Monocular Camera Calibrationcited by 34
  6. 2024SeaBird: Segmentation in Bird's View with Dice Loss Improves Monocular 3D Detection of Large Objectscited by 12
  7. 20222D GANs Meet Unsupervised Single-view 3D Reconstructioncited by 11
  8. 2024RCBEVDet++: Toward High-accuracy Radar-Camera Fusion 3D Perception Networkcited by 11
  9. 20253D-RE-GEN: 3D Reconstruction of Indoor Scenes with a Generative Frameworkcited by 10
  10. 2023Multi-view 3D Object Reconstruction and Uncertainty Modelling with Neural Shape Priorcited by 9
  11. 2025F-INR: Functional Tensor Decomposition for Implicit Neural Representationscited by 6
  12. 2024RePLAy: Remove Projective LiDAR Depthmap Artifacts via Exploiting Epipolar Geometrycited by 4
8 more
  1. 2025CHARM3R: Towards Unseen Camera Height Robust Monocular 3D Detectorcited by 4
  2. 2024Robust Shape Fitting for 3D Scene Abstractioncited by 3
  3. 2022sat2pc: Estimating Point Cloud of Building Roofs from 2D Satellite Imagescited by 2
  4. 2022Learning Implicit Functions for Dense 3D Shape Correspondence of Generic Objectscited by 2
  5. 2021Joint stereo 3D object detection and implicit surface reconstructioncited by 1
  6. 2023Advancing 3D finger knuckle recognition via deep feature learningcited by 1
  7. 20233D Semantic Subspace Traverser: Empowering 3D Generative Model with Shape Editing Capabilitycited by 1
  8. 2022Scatter Points in Space: 3D Detection from Multi-view Monocular Imagescited by 0