# Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimation

arXiv: https://arxiv.org/abs/2001.09322 · HTML: https://mlsanity.com/paper/2001.09322/

## Key facts

- Cited by 124 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jan 2020.

## Built on by (most cited first, 30 of 124)

- [FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects](https://mlsanity.com/paper/2312.08344/) (2023)
- [Where2Act: From Pixels to Actions for Articulated 3D Objects](https://mlsanity.com/paper/2101.02692/) (2021)
- [BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects](https://mlsanity.com/paper/2303.14158/) (2023)
- [FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation](https://mlsanity.com/paper/2103.02242/) (2021)
- [GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Parts](https://mlsanity.com/paper/2211.05272/) (2022)
- [FS-Net: Fast Shape-based Network for Category-Level 6D Object Pose Estimation with Decoupled Rotation Mechanism](https://mlsanity.com/paper/2103.07054/) (2021)
- [Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images](https://mlsanity.com/paper/2204.10776/) (2022)
- [RGBD Objects in the Wild: Scaling Real-World 3D Object Learning from RGB-D Videos](https://mlsanity.com/paper/2401.12592/) (2024)
- [DualPoseNet: Category-level 6D Object Pose and Size Estimation Using Dual Pose Network with Refined Learning of Pose Consistency](https://mlsanity.com/paper/2103.06526/) (2021)
- [GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Voting](https://mlsanity.com/paper/2203.07918/) (2022)
- [BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Models](https://mlsanity.com/paper/2108.00516/) (2021)
- [CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds](https://mlsanity.com/paper/2104.03437/) (2021)
- [ShAPO: Implicit Representations for Multi-Object Shape, Appearance, and Pose Optimization](https://mlsanity.com/paper/2207.13691/) (2022)
- [CenterSnap: Single-Shot Multi-Object 3D Shape Reconstruction and Categorical 6D Pose and Size Estimation](https://mlsanity.com/paper/2203.01929/) (2022)
- [Category-Level 6D Object Pose Estimation via Cascaded Relation and Recurrent Reconstruction Networks](https://mlsanity.com/paper/2108.08755/) (2021)
- [Category-Level 6D Object Pose Estimation in the Wild: A Semi-Supervised Learning Approach and A New Dataset](https://mlsanity.com/paper/2206.15436/) (2022)
- [Zero-Shot Category-Level Object Pose Estimation](https://mlsanity.com/paper/2204.03635/) (2022)
- [SAR-Net: Shape Alignment and Recovery Network for Category-level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/2106.14193/) (2021)
- [HouseCat6D -- A Large-Scale Multi-Modal Category Level 6D Object Perception Dataset with Household Objects in Realistic Scenarios](https://mlsanity.com/paper/2212.10428/) (2022)
- [SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimation](https://mlsanity.com/paper/2311.11125/) (2023)
- [HS-Pose: Hybrid Scope Feature Extraction for Category-level Object Pose Estimation](https://mlsanity.com/paper/2303.15743/) (2023)
- [Any6D: Model-free 6D Pose Estimation of Novel Objects](https://mlsanity.com/paper/2503.18673/) (2025)
- [GenPose: Generative Category-level Object Pose Estimation via Diffusion Models](https://mlsanity.com/paper/2306.10531/) (2023)
- [Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimation](https://mlsanity.com/paper/2111.00190/) (2021)
- [PhoCaL: A Multi-Modal Dataset for Category-Level Object Pose Estimation with Photometrically Challenging Objects](https://mlsanity.com/paper/2205.08811/) (2022)
- [OVE6D: Object Viewpoint Encoding for Depth-based 6D Object Pose Estimation](https://mlsanity.com/paper/2203.01072/) (2022)
- [RBP-Pose: Residual Bounding Box Projection for Category-Level Pose Estimation](https://mlsanity.com/paper/2208.00237/) (2022)
- [Single-Stage Keypoint-Based Category-Level Object Pose Estimation from an RGB Image](https://mlsanity.com/paper/2109.06161/) (2021)
- [Category-Level Metric Scale Object Shape and Pose Estimation](https://mlsanity.com/paper/2109.00326/) (2021)
- [SSP-Pose: Symmetry-Aware Shape Prior Deformation for Direct Category-Level Object Pose Estimation](https://mlsanity.com/paper/2208.06661/) (2022)

## Abstract (arXiv)

We present a novel approach to category-level 6D object pose and size estimation. To tackle intra-class shape variations, we learn canonical shape space (CASS), a unified representation for a large variety of instances of a certain object category. In particular, CASS is modeled as the latent space of a deep generative model of canonical 3D shapes with normalized pose. We train a variational auto-encoder (VAE) for generating 3D point clouds in the canonical space from an RGBD image. The VAE is trained in a cross-category fashion, exploiting the publicly available large 3D shape repositories. Since the 3D point cloud is generated in normalized pose (with actual size), the encoder of the VAE learns view-factorized RGBD embedding. It maps an RGBD image in arbitrary view into a pose-independent 3D shape representation. Object pose is then estimated via contrasting it with a pose-dependent feature of the input RGBD extracted with a separate deep neural networks. We integrate the learning of CASS and pose and size estimation into an end-to-end trainable network, achieving the state-of-the-art performance.