mlsanityData to 7 Oct 2026

Towards Self-Supervised Category-Level Object Pose and Size Estimation

arXiv 2203.02884 ↗First posted 6 Mar 2022cs.CV cs.ROcited by 11

Not on the map. Why?

Abstract (arXiv)

In this work, we tackle the challenging problem of category-level object pose and size estimation from a single depth image. Although previous fully-supervised works have demonstrated promising performance, collecting ground-truth pose labels is generally time-consuming and labor-intensive. Instead, we propose a label-free method that learns to enforce the geometric consistency between category template mesh and observed object point cloud under a self-supervision manner. Specifically, our method consists of three key components: differentiable shape deformation, registration, and rendering. In particular, shape deformation and registration are applied to the template mesh to eliminate the differences in shape, pose and scale. A differentiable renderer is then deployed to enforce geometric consistency between point clouds lifted from the rendered depth and the observed scene for self-supervision. We evaluate our approach on real-world datasets and find that our approach outperforms the simple traditional baseline by large margins while being competitive with some fully-supervised approaches.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 34 of its 48 references that are in the map. First posted on arXiv 6 Mar 2022.

Builds on

34 of 48 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  6. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  7. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  8. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  9. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  10. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  11. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 490
  12. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 352
22 more
  1. 2020PREDATOR: Registration of 3D Point Clouds with Low Overlapcited by 287
  2. 2018BOP: Benchmark for 6D Object Pose Estimationcited by 254
  3. 2018Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objectscited by 237
  4. 2017T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objectscited by 210
  5. 2019PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimationcited by 204
  6. 2017PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimationcited by 170
  7. 2020Shape Prior Deformation for Categorical 6D Object Pose and Size Estimationcited by 158
  8. 2021FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimationcited by 144
  9. 2020Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimationcited by 123
  10. 2021FS-Net: Fast Shape-based Network for Category-Level 6D Object Pose Estimation with Decoupled Rotation Mechanismcited by 121
  11. 2019HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objectscited by 110
  12. 2019Neural Cages for Detail-Preserving 3D Deformationscited by 105
  13. 2021DualPoseNet: Category-level 6D Object Pose and Size Estimation Using Dual Pose Network with Refined Learning of Pose Consistencycited by 103
  14. 2020Self6D: Self-Supervised Monocular 6D Object Pose Estimationcited by 79
  15. 2021Category-Level 6D Object Pose Estimation via Cascaded Relation and Recurrent Reconstruction Networkscited by 69
  16. 20193DN: 3D Deformation Networkcited by 60
  17. 2021Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimationcited by 45
  18. 2021UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Renderingcited by 44
  19. 2015Learning Analysis-by-Synthesis for 6D Pose Estimation in RGB-D Imagescited by 29
  20. 2020ShapeFlow: Learnable Deformations Among 3D Shapescited by 28
  21. 2017DeformNet: Free-Form Deformation Network for 3D Shape Reconstruction from a Single Imagecited by 25
  22. 2018Learning Free-Form Deformations for 3D Object Reconstructioncited by 20

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 2; 2024: 5; 2025: 2.

11 papers.

  1. 2024RGBD Objects in the Wild: Scaling Real-World 3D Object Learning from RGB-D Videoscited by 113
  2. 2022FS6D: Few-Shot 6D Pose Estimation of Novel Objectscited by 81
  3. 2024Deep Learning-Based Object Pose Estimation: A Comprehensive Surveycited by 39
  4. 2022Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wildcited by 22
  5. 2023FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objectscited by 18
  6. 2025One2Any: One-Reference 6D Pose Estimation for Any Objectcited by 17
  7. 2024OV9D: Open-Vocabulary Category-Level 9D Object Pose and Size Estimationcited by 10
  8. 2024Source-Free and Image-Only Unsupervised Domain Adaptation for Category Level Object Pose Estimationcited by 7
  9. 2023RGB-D-Based Categorical Object Pose and Shape Estimation: Methods, Datasets, and Evaluationcited by 2
  10. 2024Diffusion-Driven Self-Supervised Learning for Shape Reconstruction and Pose Estimationcited by 1
  11. 2025Universal Features Guided Zero-Shot Category-Level Object Pose Estimationcited by 0