# A Correct-and-Certify Approach to Self-Supervise Object Pose Estimators via Ensemble Self-Training

arXiv: https://arxiv.org/abs/2302.06019 · HTML: https://mlsanity.com/paper/2302.06019/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 57 references that are in the map.
- First posted on arXiv 12 Feb 2023.

## Builds on (most cited first, 30 of 45)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion](https://mlsanity.com/paper/1901.04780/) (2019)
- [CosyPose: Consistent multi-view multi-object 6D pose estimation](https://mlsanity.com/paper/2008.08465/) (2020)
- [DeepIM: Deep Iterative Matching for 6D Pose Estimation](https://mlsanity.com/paper/1804.00175/) (2018)
- [TEASER: Fast and Certifiable Point Cloud Registration](https://mlsanity.com/paper/2001.07715/) (2020)
- [BOP: Benchmark for 6D Object Pose Estimation](https://mlsanity.com/paper/1808.08319/) (2018)
- [PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Sampling](https://mlsanity.com/paper/2003.00492/) (2020)
- [Differentiation of Blackbox Combinatorial Solvers](https://mlsanity.com/paper/1912.02175/) (2019)
- [T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objects](https://mlsanity.com/paper/1701.05498/) (2017)
- [BOP Challenge 2020 on 6D Object Localization](https://mlsanity.com/paper/2009.07378/) (2020)
- [SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver](https://mlsanity.com/paper/1905.12149/) (2019)
- [Modeling Point Clouds with Self-Attention and Gumbel Subset Sampling](https://mlsanity.com/paper/1904.03375/) (2019)
- [Self-supervised 6D Object Pose Estimation for Robot Manipulation](https://mlsanity.com/paper/1909.10159/) (2019)
- [Graduated Non-Convexity for Robust Spatial Perception: From Non-Minimal Solvers to Global Outlier Rejection](https://mlsanity.com/paper/1909.08605/) (2019)
- [6-DoF Object Pose from Semantic Keypoints](https://mlsanity.com/paper/1703.04670/) (2017)
- [Self6D: Self-Supervised Monocular 6D Object Pose Estimation](https://mlsanity.com/paper/2004.06468/) (2020)
- [Deep Declarative Networks: A New Hope](https://mlsanity.com/paper/1909.04866/) (2019)
- [Autolabeling 3D Objects with Differentiable Rendering of SDF Shape Priors](https://mlsanity.com/paper/1911.11288/) (2019)
- [SampleNet: Differentiable Point Cloud Sampling](https://mlsanity.com/paper/1912.03663/) (2019)
- [Multi-view Self-supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challenge](https://mlsanity.com/paper/1609.09475/) (2016)
- [Theseus: A Library for Differentiable Nonlinear Optimization](https://mlsanity.com/paper/2207.09442/) (2022)
- [Satellite Pose Estimation with Deep Landmark Regression and Nonlinear Pose Refinement](https://mlsanity.com/paper/1908.11542/) (2019)

## Built on by (most cited first, 6 of 6)

- [CRISP: Object Pose and Shape Estimation with Test-Time Adaptation](https://mlsanity.com/paper/2412.01052/) (2024)
- [Certifying Robustness of Learning-Based Keypoint Detection and Pose Estimation Methods](https://mlsanity.com/paper/2408.00117/) (2024)
- [Picasso: Holistic Scene Reconstruction with Physics-Constrained Sampling](https://mlsanity.com/paper/2602.08058/) (2026)
- [Test-Time Certifiable Self-Supervision to Bridge the Sim2Real Gap in Event-Based Satellite Pose Estimation](https://mlsanity.com/paper/2409.06240/) (2024)
- [Box Pose and Shape Estimation and Domain Adaptation for Large-Scale Warehouse Automation](https://mlsanity.com/paper/2507.00984/) (2025)
- [From Decoupled to Coupled: Robustness Verification for Learning-based Keypoint Detection with Joint Specifications](https://mlsanity.com/paper/2603.05604/) (2026)

## Abstract (arXiv)

Real-world robotics applications demand object pose estimation methods that work reliably across a variety of scenarios. Modern learning-based approaches require large labeled datasets and tend to perform poorly outside the training domain. Our first contribution is to develop a robust corrector module that corrects pose estimates using depth information, thus enabling existing methods to better generalize to new test domains; the corrector operates on semantic keypoints (but is also applicable to other pose estimators) and is fully differentiable. Our second contribution is an ensemble self-training approach that simultaneously trains multiple pose estimators in a self-supervised manner. Our ensemble self-training architecture uses the robust corrector to refine the output of each pose estimator; then, it evaluates the quality of the outputs using observable correctness certificates; finally, it uses the observably correct outputs for further training, without requiring external supervision. As an additional contribution, we propose small improvements to a regression-based keypoint detection architecture, to enhance its robustness to outliers; these improvements include a robust pooling scheme and a robust centroid computation. Experiments on the YCBV and TLESS datasets show the proposed ensemble self-training outperforms fully supervised baselines while not requiring 3D annotations on real data.