# Weakly Supervised Learning of Keypoints for 6D Object Pose Estimation

arXiv: https://arxiv.org/abs/2203.03498 · HTML: https://mlsanity.com/paper/2203.03498/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 43 references that are in the map.
- First posted on arXiv 7 Mar 2022.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1812.11788/) (2018)
- [DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion](https://mlsanity.com/paper/1901.04780/) (2019)
- [Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks](https://mlsanity.com/paper/1612.05424/) (2016)
- [Integral Human Pose Regression](https://mlsanity.com/paper/1711.08229/) (2017)
- [SSD-6D: Making RGB-based 3D detection and 6D pose estimation great again](https://mlsanity.com/paper/1711.10006/) (2017)
- [BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth](https://mlsanity.com/paper/1703.10896/) (2017)
- [Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation](https://mlsanity.com/paper/1908.07433/) (2019)
- [Real-Time Seamless Single Shot 6D Object Pose Prediction](https://mlsanity.com/paper/1711.08848/) (2017)
- [PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1911.04231/) (2019)
- [BOP Challenge 2020 on 6D Object Localization](https://mlsanity.com/paper/2009.07378/) (2020)
- [DPOD: 6D Pose Object Detector and Refiner](https://mlsanity.com/paper/1902.11020/) (2019)
- [Implicit 3D Orientation Learning for 6D Object Detection from RGB Images](https://mlsanity.com/paper/1902.01275/) (2019)
- [Learnable Triangulation of Human Pose](https://mlsanity.com/paper/1905.05754/) (2019)
- [HybridPose: 6D Object Pose Estimation under Hybrid Representations](https://mlsanity.com/paper/2001.01869/) (2020)
- [Segmentation-driven 6D Object Pose Estimation](https://mlsanity.com/paper/1812.02541/) (2018)
- [Self6D: Self-Supervised Monocular 6D Object Pose Estimation](https://mlsanity.com/paper/2004.06468/) (2020)
- [Making Deep Heatmaps Robust to Partial Occlusions for 3D Object Pose Estimation](https://mlsanity.com/paper/1804.03959/) (2018)
- [Explaining the Ambiguity of Object Detection and 6D Pose From Visual Data](https://mlsanity.com/paper/1812.00287/) (2018)
- [A Unified Framework for Multi-View Multi-Class Object Pose Estimation](https://mlsanity.com/paper/1803.08103/) (2018)
- [DeceptionNet: Network-Driven Domain Randomization](https://mlsanity.com/paper/1904.02750/) (2019)
- [Deep Learning of Local RGB-D Patches for 3D Object Detection and 6D Pose Estimation](https://mlsanity.com/paper/1607.06038/) (2016)
- [Feature Mapping for Learning Fast and Accurate 3D Pose Inference from Synthetic Images](https://mlsanity.com/paper/1712.03904/) (2017)
- [Neural Object Learning for 6D Pose Estimation Using a Few Cluttered Images](https://mlsanity.com/paper/2005.03717/) (2020)
- [Domain Transfer for 3D Pose Estimation from Color Images without Manual Annotations](https://mlsanity.com/paper/1810.03707/) (2018)
- [Seeing Beyond Appearance - Mapping Real Images into Geometrical Domains for Unsupervised CAD-based Recognition](https://mlsanity.com/paper/1810.04158/) (2018)

## Abstract (arXiv)

State-of-the-art approaches for 6D object pose estimation require large amounts of labeled data to train the deep networks. However, the acquisition of 6D object pose annotations is tedious and labor-intensive in large quantity. To alleviate this problem, we propose a weakly supervised 6D object pose estimation approach based on 2D keypoint detection. Our method trains only on image pairs with known relative transformations between their viewpoints. Specifically, we assign a set of arbitrarily chosen 3D keypoints to represent each unknown target 3D object and learn a network to detect their 2D projections that comply with the relative camera viewpoints. During inference, our network first infers the 2D keypoints from the query image and a given labeled reference image. We then use these 2D keypoints and the arbitrarily chosen 3D keypoints retained from training to infer the 6D object pose. Extensive experiments demonstrate that our approach achieves comparable performance with state-of-the-art fully supervised approaches.