mlsanityData to 7 Oct 2026

CenterSnap: Single-Shot Multi-Object 3D Shape Reconstruction and Categorical 6D Pose and Size Estimation

arXiv 2203.01929 ↗First posted 3 Mar 2022cs.CV cs.LG cs.ROcited by 69

Not on the map. Why?

Abstract (arXiv)

This paper studies the complex task of simultaneous multi-object 3D reconstruction, 6D pose and size estimation from a single-view RGB-D observation. In contrast to instance-level pose estimation, we focus on a more challenging problem where CAD models are not available at inference time. Existing approaches mainly follow a complex multi-stage pipeline which first localizes and detects each object instance in the image and then regresses to either their 3D meshes or 6D poses. These approaches suffer from high-computational cost and low performance in complex multi-object scenarios, where occlusions can be present. Hence, we present a simple one-stage approach to predict both the 3D shape and estimate the 6D pose and size jointly in a bounding-box free manner. In particular, our method treats object instances as spatial centers where each center denotes the complete shape of an object along with its 6D pose and size. Through this per-pixel representation, our approach can reconstruct in real-time (40 FPS) multiple novel object instances and predict their 6D pose and sizes in a single-forward pass. Through extensive experiments, we demonstrate that our approach significantly outperforms all shape completion and categorical 6D pose and size estimation baselines on multi-object ShapeNet and NOCS datasets respectively with a 12.6% absolute improvement in mAP for 6D pose for novel real-world object instances.

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). Builds on 57 of its 68 references that are in the map. First posted on arXiv 3 Mar 2022.

Builds on

57 of 68 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2017Mask R-CNNcited by 7,259
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  7. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  8. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  9. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  10. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  11. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  12. 2019Objects as Pointscited by 1,172
38 more
  1. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  2. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  3. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  4. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
  5. 2018Open3D: A Modern Library for 3D Data Processingcited by 717
  6. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  7. 2019CenterNet: Keypoint Triplets for Object Detectioncited by 654
  8. 2019Panoptic Feature Pyramid Networkscited by 644
  9. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  10. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  11. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  12. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 490
  13. 2020Tracking Objects as Pointscited by 483
  14. 2017Neural 3D Mesh Renderercited by 437
  15. 2018PVNet: Pixel-wise Voting Network for 6DoF Pose Estimationcited by 385
  16. 2018PCN: Point Completion Networkcited by 382
  17. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 352
  18. 2020Conditional Convolutions for Instance Segmentationcited by 314
  19. 2019SOLO: Segmenting Objects by Locationscited by 289
  20. 2017SSD-6D: Making RGB-based 3D detection and 6D pose estimation great againcited by 243
  21. 2017BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depthcited by 225
  22. 2020Monocular, One-stage, Regression of Multiple 3D Peoplecited by 216
  23. 2017Real-Time Seamless Single Shot 6D Object Pose Predictioncited by 210
  24. 2019Implicit 3D Orientation Learning for 6D Object Detection from RGB Imagescited by 170
  25. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 170
  26. 2020Shape Prior Deformation for Categorical 6D Object Pose and Size Estimationcited by 158
  27. 2020Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimationcited by 123
  28. 2021Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representationscited by 107
  29. 2020Category Level Object Pose Estimation via Neural Analysis-by-Synthesiscited by 100
  30. 2019Single-Stage Multi-Person Pose Machinescited by 95
  31. 2021Holistic 3D Scene Understanding from a Single Image with Implicit Representationcited by 94
  32. 2021BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Modelscited by 87
  33. 2016Shape Completion Enabled Robotic Graspingcited by 79
  34. 20196-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypointscited by 78
  35. 2020Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrievecited by 65
  36. 2020FroDO: From Detections to 3D Objectscited by 51
  37. 2021Category-Level Metric Scale Object Shape and Pose Estimationcited by 40
  38. 2016Deep Learning of Local RGB-D Patches for 3D Object Detection and 6D Pose Estimationcited by 34

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 19; 2024: 18; 2025: 14; 2026: 8.

69 papers.

Sort:

By area:

  1. 2023GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondencecited by 113
  2. 2022ShAPO: Implicit Representations for Multi-Object Shape, Appearance, and Pose Optimizationcited by 70
  3. 2023CARTO: Category and Joint Agnostic Reconstruction of ARTiculated Objectscited by 64
  4. 2023NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenescited by 53
  5. 2023SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimationcited by 52
  6. 2022HouseCat6D -- A Large-Scale Multi-Modal Category Level 6D Object Perception Dataset with Household Objects in Realistic Scenarioscited by 51
  7. 2023DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Imagecited by 48
  8. 2023HANDAL: A Dataset of Real-World Manipulable Object Categories with Pose Annotations, Affordances, and Reconstructionscited by 47
  9. 2022SSP-Pose: Symmetry-Aware Shape Prior Deformation for Direct Category-Level Object Pose Estimationcited by 41
  10. 2023VI-Net: Boosting Category-level 6D Object Pose Estimation via Learning Decoupled Rotations on the Spherical Representationscited by 40
  11. 2024Deep Learning-Based Object Pose Estimation: A Comprehensive Surveycited by 39
  12. 2024DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimationcited by 24
38 more
  1. 2022Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wildcited by 22
  2. 2024OmniNOCS: A unified NOCS dataset and model for 3D lifting of 2D objectscited by 22
  3. 2024Neural Fields in Robotics: A Surveycited by 21
  4. 2024NeRF-MAE: Masked AutoEncoders for Self-Supervised 3D Representation Learning for Neural Radiance Fieldscited by 20
  5. 2022Generative Category-Level Shape and Pose Estimation with Semantic Primitivescited by 19
  6. 2023PanoContext-Former: Panoramic Total Scene Understanding with a Transformercited by 19
  7. 2022StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCScited by 18
  8. 2023FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objectscited by 18
  9. 2025One2Any: One-Reference 6D Pose Estimation for Any Objectcited by 17
  10. 2025GCE-Pose: Global Context Enhancement for Category-level Object Pose Estimationcited by 16
  11. 2023Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environmentscited by 15
  12. 2022TransNet: Category-Level Transparent Object Pose Estimationcited by 13
  13. 2023CenterGrasp: Object-Aware Implicit Representation Learning for Simultaneous Shape Reconstruction and 6-DoF Grasp Estimationcited by 13
  14. 2025Detection Based Part-level Articulated Object Reconstruction from Single RGBD Imagecited by 13
  15. 2025ZeroGrasp: Zero-Shot Shape Reconstruction Enabled Robotic Graspingcited by 13
  16. 2025SplArt: Articulation Estimation and Part-Level Reconstruction with 3D Gaussian Splattingcited by 13
  17. 2024Omni6D: Large-Vocabulary 3D Object Dataset for Category-Level 6D Object Pose Estimationcited by 11
  18. 2025See4D: Pose-Free 4D Generation via Auto-Regressive Video Inpaintingcited by 11
  19. 2024OV9D: Open-Vocabulary Category-Level 9D Object Pose and Size Estimationcited by 10
  20. 2024Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modelingcited by 10
  21. 2023O2-Recon: Completing 3D Reconstruction of Occluded Objects in the Scene with a Pre-trained 2D Diffusion Modelcited by 8
  22. 2024OP-Align: Object-level and Part-level Alignment for Self-supervised Category-level Articulated Object Pose Estimationcited by 8
  23. 2022SDFEst: Categorical Pose and Shape Estimation of Objects from RGB-D using Signed Distance Fieldscited by 7
  24. 2022CPPF++: Uncertainty-Aware Sim2Real Object Pose Estimation by Vote Aggregationcited by 6
  25. 2023DTF-Net: Category-Level Pose Estimation and Shape Reconstruction via Deformable Template Fieldcited by 5
  26. 2023LASA: Instance Reconstruction from Real Scans using A Large-scale Aligned Shape Annotation Datasetcited by 5
  27. 2024Category-level Object Detection, Pose Estimation and Reconstruction from Stereo Imagescited by 5
  28. 2025GIVEPose: Gradual Intra-class Variation Elimination for RGB-based Category-Level Object Pose Estimationcited by 5
  29. 2026Reconstruction by Generation: 3D Multi-Object Scene Reconstruction from Sparse Observationscited by 5
  30. 2024Extreme Two-View Geometry From Object Poses with Diffusion Modelscited by 4
  31. 2025MonoDiff9D: Monocular Category-Level 9D Object Pose Estimation via Diffusion Modelcited by 4
  32. 2023RIC: Rotate-Inpaint-Complete for Generalizable Scene Reconstructioncited by 3
  33. 2024Towards Real-World Aerial Vision Guidance with Categorical 6D Pose Trackercited by 3
  34. 2025Unified Category-Level Object Detection and Pose Estimation from RGB Images using 3D Prototypescited by 3
  35. 2023RGB-D-Based Categorical Object Pose and Shape Estimation: Methods, Datasets, and Evaluationcited by 2
  36. 2023Real-time Simultaneous Multi-Object 3D Shape Reconstruction, 6DoF Pose Estimation and Dense Grasp Predictioncited by 2
  37. 2025Beyond 'Templates': Category-Agnostic Object Pose, Size, and Shape Estimation from a Single Viewcited by 2
  38. 2023MV-ROPE: Multi-view Constraints for Robust Category-level Object Pose and Size Estimationcited by 1

50 most cited of 69 All 69 (JSON)