# DSC-PoseNet: Learning 6DoF Object Pose Estimation via Dual-scale Consistency

arXiv: https://arxiv.org/abs/2104.03658 · HTML: https://mlsanity.com/paper/2104.03658/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 52 references that are in the map.
- First posted on arXiv 8 Apr 2021.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1812.11788/) (2018)
- [CosyPose: Consistent multi-view multi-object 6D pose estimation](https://mlsanity.com/paper/2008.08465/) (2020)
- [DeepIM: Deep Iterative Matching for 6D Pose Estimation](https://mlsanity.com/paper/1804.00175/) (2018)
- [BOP: Benchmark for 6D Object Pose Estimation](https://mlsanity.com/paper/1808.08319/) (2018)
- [SSD-6D: Making RGB-based 3D detection and 6D pose estimation great again](https://mlsanity.com/paper/1711.10006/) (2017)
- [BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth](https://mlsanity.com/paper/1703.10896/) (2017)
- [Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation](https://mlsanity.com/paper/1908.07433/) (2019)
- [Real-Time Seamless Single Shot 6D Object Pose Prediction](https://mlsanity.com/paper/1711.08848/) (2017)
- [DPOD: 6D Pose Object Detector and Refiner](https://mlsanity.com/paper/1902.11020/) (2019)
- [Implicit 3D Orientation Learning for 6D Object Detection from RGB Images](https://mlsanity.com/paper/1902.01275/) (2019)
- [EPOS: Estimating 6D Pose of Objects with Symmetries](https://mlsanity.com/paper/2004.00605/) (2020)
- [SOSNet: Second Order Similarity Regularization for Local Descriptor Learning](https://mlsanity.com/paper/1904.05019/) (2019)
- [HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objects](https://mlsanity.com/paper/1904.03167/) (2019)
- [HybridPose: 6D Object Pose Estimation under Hybrid Representations](https://mlsanity.com/paper/2001.01869/) (2020)
- [Weakly-Supervised Salient Object Detection via Scribble Annotations](https://mlsanity.com/paper/2003.07685/) (2020)
- [Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views](https://mlsanity.com/paper/1505.05641/) (2015)
- [Single-Stage 6D Object Pose Estimation](https://mlsanity.com/paper/1911.08324/) (2019)
- [Numerical Coordinate Regression with Convolutional Neural Networks](https://mlsanity.com/paper/1801.07372/) (2018)
- [Explaining the Ambiguity of Object Detection and 6D Pose From Visual Data](https://mlsanity.com/paper/1812.00287/) (2018)
- [Photorealistic Image Synthesis for Object Instance Detection](https://mlsanity.com/paper/1902.03334/) (2019)
- [End-to-End Learnable Geometric Vision by Backpropagating PnP Optimization](https://mlsanity.com/paper/1909.06043/) (2019)
- [Viewpoints and Keypoints](https://mlsanity.com/paper/1411.6067/) (2014)
- [Deep Model-Based 6D Pose Refinement in RGB](https://mlsanity.com/paper/1810.03065/) (2018)

## Built on by (most cited first, 14 of 14)

- [Deep Learning-Based Object Pose Estimation: A Comprehensive Survey](https://mlsanity.com/paper/2405.07801/) (2024)
- [Deep Learning on Monocular Object Pose Detection and Tracking: A Comprehensive Overview](https://mlsanity.com/paper/2105.14291/) (2021)
- [TexPose: Neural Texture Learning for Self-Supervised 6D Object Pose Estimation](https://mlsanity.com/paper/2212.12902/) (2022)
- [FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objects](https://mlsanity.com/paper/2310.12974/) (2023)
- [TransHuman: A Transformer-based Human Representation for Generalizable Neural Human Rendering](https://mlsanity.com/paper/2307.12291/) (2023)
- [JOTR: 3D Joint Contrastive Learning with Transformers for Occluded Human Mesh Recovery](https://mlsanity.com/paper/2307.16377/) (2023)
- [Sim2Real Instance-Level Style Transfer for 6D Pose Estimation](https://mlsanity.com/paper/2203.02069/) (2022)
- [Pseudo Flow Consistency for Self-Supervised 6D Object Pose Estimation](https://mlsanity.com/paper/2308.10016/) (2023)
- [Keypoint Cascade Voting for Point Cloud Based 6DoF Pose Estimation](https://mlsanity.com/paper/2210.08123/) (2022)
- [Distortion Resilience for Goal-Oriented Semantic Communication](https://mlsanity.com/paper/2309.14587/) (2023)
- [Pseudo-keypoint RKHS Learning for Self-supervised 6DoF Pose Estimation](https://mlsanity.com/paper/2311.09500/) (2023)
- [CASAPose: Class-Adaptive and Semantic-Aware Multi-Object Pose Estimation](https://mlsanity.com/paper/2210.05318/) (2022)
- [Manifold-Aware Self-Training for Unsupervised Domain Adaptation on Regressing 6D Object Pose](https://mlsanity.com/paper/2305.10808/) (2023)
- [Weakly Supervised 3D Object Detection with Multi-Stage Generalization](https://mlsanity.com/paper/2306.05418/) (2023)

## Abstract (arXiv)

Compared to 2D object bounding-box labeling, it is very difficult for humans to annotate 3D object poses, especially when depth images of scenes are unavailable. This paper investigates whether we can estimate the object poses effectively when only RGB images and 2D object annotations are given. To this end, we present a two-step pose estimation framework to attain 6DoF object poses from 2D object bounding-boxes. In the first step, the framework learns to segment objects from real and synthetic data in a weakly-supervised fashion, and the segmentation masks will act as a prior for pose estimation. In the second step, we design a dual-scale pose estimation network, namely DSC-PoseNet, to predict object poses by employing a differential renderer. To be specific, our DSC-PoseNet firstly predicts object poses in the original image scale by comparing the segmentation masks and the rendered visible object masks. Then, we resize object regions to a fixed scale to estimate poses once again. In this fashion, we eliminate large scale variations and focus on rotation estimation, thus facilitating pose estimation. Moreover, we exploit the initial pose estimation to generate pseudo ground-truth to train our DSC-PoseNet in a self-supervised manner. The estimation results in these two scales are ensembled as our final pose estimation. Extensive experiments on widely-used benchmarks demonstrate that our method outperforms state-of-the-art models trained on synthetic data by a large margin and even is on par with several fully-supervised methods.