# YOLOPose: Transformer-based Multi-Object 6D Pose Estimation using Keypoint Regression

arXiv: https://arxiv.org/abs/2205.02536 · HTML: https://mlsanity.com/paper/2205.02536/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 31 references that are in the map.
- First posted on arXiv 5 May 2022.

## Builds on (most cited first, 23 of 23)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1812.11788/) (2018)
- [CosyPose: Consistent multi-view multi-object 6D pose estimation](https://mlsanity.com/paper/2008.08465/) (2020)
- [DeepIM: Deep Iterative Matching for 6D Pose Estimation](https://mlsanity.com/paper/1804.00175/) (2018)
- [BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth](https://mlsanity.com/paper/1703.10896/) (2017)
- [Real-Time Seamless Single Shot 6D Object Pose Prediction](https://mlsanity.com/paper/1711.08848/) (2017)
- [BOP Challenge 2020 on 6D Object Localization](https://mlsanity.com/paper/2009.07378/) (2020)
- [GDRNPP: A Geometry-guided and Fully Learning-based Object Pose Estimator](https://mlsanity.com/paper/2102.12145/) (2021)
- [End-to-end people detection in crowded scenes](https://mlsanity.com/paper/1506.04878/) (2015)
- [Segmentation-driven 6D Object Pose Estimation](https://mlsanity.com/paper/1812.02541/) (2018)
- [Single-Stage 6D Object Pose Estimation](https://mlsanity.com/paper/1911.08324/) (2019)
- [Making Deep Heatmaps Robust to Partial Occlusions for 3D Object Pose Estimation](https://mlsanity.com/paper/1804.03959/) (2018)
- [End-to-End Learnable Geometric Vision by Backpropagating PnP Optimization](https://mlsanity.com/paper/1909.06043/) (2019)
- [Deep Model-Based 6D Pose Refinement in RGB](https://mlsanity.com/paper/1810.03065/) (2018)
- [Inductive Bias of Deep Convolutional Networks through Pooling Geometry](https://mlsanity.com/paper/1605.06743/) (2016)
- [PFRL: Pose-Free Reinforcement Learning for 6D Pose Estimation](https://mlsanity.com/paper/2102.12096/) (2021)
- [Exploring intermediate representation for monocular vehicle pose estimation](https://mlsanity.com/paper/2011.08464/) (2020)
- [Refining 6D Object Pose Predictions using Abstract Render-and-Compare](https://mlsanity.com/paper/1910.03412/) (2019)
- [Robust 6D Object Pose Estimation in Cluttered Scenes using Semantic Segmentation and Pose Regression Networks](https://mlsanity.com/paper/1810.03410/) (2018)

## Built on by (most cited first, 17 of 17)

- [Confronting Ambiguity in 6D Object Pose Estimation via Score-Based Diffusion on SE(3)](https://mlsanity.com/paper/2305.15873/) (2023)
- [CRT-6D: Fast 6D Object Pose Estimation with Cascaded Refinement Transformers](https://mlsanity.com/paper/2210.11718/) (2022)
- [Implicit-Zoo: A Large-Scale Dataset of Neural Implicit Functions for 2D Images and 3D Scenes](https://mlsanity.com/paper/2406.17438/) (2024)
- [TransPose: 6D Object Pose Estimation with Geometry-Aware Transformer](https://mlsanity.com/paper/2310.16279/) (2023)
- [GBOT: Graph-Based 3D Object Tracking for Augmented Reality-Assisted Assembly Guidance](https://mlsanity.com/paper/2402.07677/) (2024)
- [ASDF: Assembly State Detection Utilizing Late Fusion by Integrating 6D Pose Estimation](https://mlsanity.com/paper/2403.16400/) (2024)
- [Video based Object 6D Pose Estimation using Transformers](https://mlsanity.com/paper/2210.13540/) (2022)
- [RDPN6D: Residual-based Dense Point-wise Network for 6Dof Object Pose Estimation Based on RGB-D Images](https://mlsanity.com/paper/2405.08483/) (2024)
- [External Camera-based Mobile Robot Pose Estimation for Collaborative Perception with Smart Edge Sensors](https://mlsanity.com/paper/2303.03797/) (2023)
- [YOLOPose V2: Understanding and Improving Transformer-based 6D Pose Estimation](https://mlsanity.com/paper/2307.11550/) (2023)
- [Learning Implicit Probability Distribution Functions for Symmetric Orientation Estimation from RGB Images Without Pose Labels](https://mlsanity.com/paper/2211.11394/) (2022)
- [Deep Transformer Network for Monocular Pose Estimation of Shipborne Unmanned Aerial Vehicle](https://mlsanity.com/paper/2406.09260/) (2024)
- [KVN: Keypoints Voting Network with Differentiable RANSAC for Stereo Pose Estimation](https://mlsanity.com/paper/2307.11543/) (2023)
- [Efficient Multi-Object Pose Estimation using Multi-Resolution Deformable Attention and Query Aggregation](https://mlsanity.com/paper/2312.08268/) (2023)
- [Sparse Color-Code Net: Real-Time RGB-Based 6D Object Pose Estimation on Edge Devices](https://mlsanity.com/paper/2406.02977/) (2024)
- [A Novel Convolution and Attention Mechanism-based Model for 6D Object Pose Estimation](https://mlsanity.com/paper/2501.01993/) (2024)
- [TinyDETR-Pose: Towards End-to-End Real-Time Single-Stage 6DoF Object Pose Estimation with Lightweight Transformers](https://mlsanity.com/paper/2608.15297/) (2026)

## Abstract (arXiv)

6D object pose estimation is a crucial prerequisite for autonomous robot manipulation applications. The state-of-the-art models for pose estimation are convolutional neural network (CNN)-based. Lately, Transformers, an architecture originally proposed for natural language processing, is achieving state-of-the-art results in many computer vision tasks as well. Equipped with the multi-head self-attention mechanism, Transformers enable simple single-stage end-to-end architectures for learning object detection and 6D object pose estimation jointly. In this work, we propose YOLOPose (short form for You Only Look Once Pose estimation), a Transformer-based multi-object 6D pose estimation method based on keypoint regression. In contrast to the standard heatmaps for predicting keypoints in an image, we directly regress the keypoints. Additionally, we employ a learnable orientation estimation module to predict the orientation from the keypoints. Along with a separate translation estimation module, our model is end-to-end differentiable. Our method is suitable for real-time applications and achieves results comparable to state-of-the-art methods.