# RGB-based Category-level Object Pose Estimation via Decoupled Metric Scale Recovery

arXiv: https://arxiv.org/abs/2309.10255 · HTML: https://mlsanity.com/paper/2309.10255/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 19 Sep 2023.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans](https://mlsanity.com/paper/2110.04994/) (2021)
- [Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation](https://mlsanity.com/paper/2007.08454/) (2020)
- [Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/2001.09322/) (2020)
- [FS-Net: Fast Shape-based Network for Category-Level 6D Object Pose Estimation with Decoupled Rotation Mechanism](https://mlsanity.com/paper/2103.07054/) (2021)
- [3D Common Corruptions and Data Augmentation](https://mlsanity.com/paper/2203.01441/) (2022)
- [DualPoseNet: Category-level 6D Object Pose and Size Estimation Using Dual Pose Network with Refined Learning of Pose Consistency](https://mlsanity.com/paper/2103.06526/) (2021)
- [Category Level Object Pose Estimation via Neural Analysis-by-Synthesis](https://mlsanity.com/paper/2008.08145/) (2020)
- [GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Voting](https://mlsanity.com/paper/2203.07918/) (2022)
- [Category-Level 6D Object Pose and Size Estimation using Self-Supervised Deep Prior Deformation Networks](https://mlsanity.com/paper/2207.05444/) (2022)
- [Category-Level 6D Object Pose Estimation via Cascaded Relation and Recurrent Reconstruction Networks](https://mlsanity.com/paper/2108.08755/) (2021)
- [RePOSE: Fast 6D Object Pose Refinement via Deep Texture Rendering](https://mlsanity.com/paper/2104.00633/) (2021)
- [SAR-Net: Shape Alignment and Recovery Network for Category-level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/2106.14193/) (2021)
- [Category-Level Metric Scale Object Shape and Pose Estimation](https://mlsanity.com/paper/2109.00326/) (2021)
- [RNNPose: Recurrent 6-DoF Object Pose Refinement with Robust Correspondence Field Estimation and Pose Optimization](https://mlsanity.com/paper/2203.12870/) (2022)
- [Object Level Depth Reconstruction for Category Level 6D Object Pose Estimation From Monocular RGB Image](https://mlsanity.com/paper/2204.01586/) (2022)
- [Vision-based Robotic Grasping From Object Localization, Object Pose Estimation to Grasp Estimation for Parallel Grippers: A Review](https://mlsanity.com/paper/1905.06658/) (2019)
- [6D Object Pose Estimation with Depth Images: A Seamless Approach for Robotic Interaction and Augmented Reality](https://mlsanity.com/paper/1709.01459/) (2017)
- [RGB-D-Based Categorical Object Pose and Shape Estimation: Methods, Datasets, and Evaluation](https://mlsanity.com/paper/2301.08147/) (2023)

## Built on by (most cited first, 14 of 14)

- [Deep Learning-Based Object Pose Estimation: A Comprehensive Survey](https://mlsanity.com/paper/2405.07801/) (2024)
- [InterTrack: Tracking Human Object Interaction without Object Templates](https://mlsanity.com/paper/2408.13953/) (2024)
- [LaPose: Laplacian Mixture Shape Modeling for RGB-Based Category-Level Object Pose Estimation](https://mlsanity.com/paper/2409.15727/) (2024)
- [Orientation Matters: Making 3D Generative Models Orientation-Aligned](https://mlsanity.com/paper/2506.08640/) (2025)
- [GIVEPose: Gradual Intra-class Variation Elimination for RGB-based Category-Level Object Pose Estimation](https://mlsanity.com/paper/2503.15110/) (2025)
- [MonoDiff9D: Monocular Category-Level 9D Object Pose Estimation via Diffusion Model](https://mlsanity.com/paper/2504.10433/) (2025)
- [Unified Category-Level Object Detection and Pose Estimation from RGB Images using 3D Prototypes](https://mlsanity.com/paper/2508.02157/) (2025)
- [You Only Pose Once: A Minimalist's Detection Transformer for Monocular RGB Category-level 9D Multi-Object Pose Estimation](https://mlsanity.com/paper/2508.14965/) (2025)
- [Category Level 6D Object Pose Estimation from a Single RGB Image using Diffusion](https://mlsanity.com/paper/2412.11420/) (2024)
- [PoseIDON: 6DoF Pose Estimation with Foundation Model Features for Marine Sediment Burial Mapping](https://mlsanity.com/paper/2506.10386/) (2025)
- [RCGNet: RGB-based Category-Level 6D Object Pose Estimation with Geometric Guidance](https://mlsanity.com/paper/2508.13623/) (2025)
- [Object Pose Transformer: Unifying Unseen Object Pose Estimation](https://mlsanity.com/paper/2603.23370/) (2026)
- [Emergence of a Shared Canonical Object Frame from In-the-Wild Videos](https://mlsanity.com/paper/2606.30058/) (2026)
- [SchemaPose: RGB-Based Category-Level Pose Estimation with Parametric Category Schema](https://mlsanity.com/paper/2610.18374/) (2026)

## Abstract (arXiv)

While showing promising results, recent RGB-D camera-based category-level object pose estimation methods have restricted applications due to the heavy reliance on depth sensors. RGB-only methods provide an alternative to this problem yet suffer from inherent scale ambiguity stemming from monocular observations. In this paper, we propose a novel pipeline that decouples the 6D pose and size estimation to mitigate the influence of imperfect scales on rigid transformations. Specifically, we leverage a pre-trained monocular estimator to extract local geometric information, mainly facilitating the search for inlier 2D-3D correspondence. Meanwhile, a separate branch is designed to directly recover the metric scale of the object based on category-level statistics. Finally, we advocate using the RANSAC-PnP algorithm to robustly solve for 6D object pose. Extensive experiments have been conducted on both synthetic and real datasets, demonstrating the superior performance of our method over previous state-of-the-art RGB-based approaches, especially in terms of rotation accuracy. Code: https://github.com/goldoak/DMSR.