# PoET: Pose Estimation Transformer for Single-View, Multi-Object 6D Pose Estimation

arXiv: https://arxiv.org/abs/2211.14125 · HTML: https://mlsanity.com/paper/2211.14125/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 25 Nov 2022.

## Builds on (most cited first, 22 of 22)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1812.11788/) (2018)
- [CosyPose: Consistent multi-view multi-object 6D pose estimation](https://mlsanity.com/paper/2008.08465/) (2020)
- [DeepIM: Deep Iterative Matching for 6D Pose Estimation](https://mlsanity.com/paper/1804.00175/) (2018)
- [SSD-6D: Making RGB-based 3D detection and 6D pose estimation great again](https://mlsanity.com/paper/1711.10006/) (2017)
- [BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth](https://mlsanity.com/paper/1703.10896/) (2017)
- [Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation](https://mlsanity.com/paper/1908.07433/) (2019)
- [Scaled-YOLOv4: Scaling Cross Stage Partial Network](https://mlsanity.com/paper/2011.08036/) (2020)
- [BOP Challenge 2020 on 6D Object Localization](https://mlsanity.com/paper/2009.07378/) (2020)
- [EPOS: Estimating 6D Pose of Objects with Symmetries](https://mlsanity.com/paper/2004.00605/) (2020)
- [6-DoF Object Pose from Semantic Keypoints](https://mlsanity.com/paper/1703.04670/) (2017)
- [A Unified Framework for Multi-View Multi-Class Object Pose Estimation](https://mlsanity.com/paper/1803.08103/) (2018)
- [Learning Analysis-by-Synthesis for 6D Pose Estimation in RGB-D Images](https://mlsanity.com/paper/1508.04546/) (2015)
- [Global Hypothesis Generation for 6D Object Pose Estimation](https://mlsanity.com/paper/1612.02287/) (2016)
- [T6D-Direct: Transformers for Multi-Object 6D Pose Direct Regression](https://mlsanity.com/paper/2109.10948/) (2021)
- [SilhoNet: An RGB Method for 6D Object Pose Estimation](https://mlsanity.com/paper/1809.06893/) (2018)
- [Virtual Training for a Real Application: Accurate Object-Robot Relative Localization without Calibration](https://mlsanity.com/paper/1902.02711/) (2019)

## Built on by (most cited first, 13 of 13)

- [Implicit-Zoo: A Large-Scale Dataset of Neural Implicit Functions for 2D Images and 3D Scenes](https://mlsanity.com/paper/2406.17438/) (2024)
- [GBOT: Graph-Based 3D Object Tracking for Augmented Reality-Assisted Assembly Guidance](https://mlsanity.com/paper/2402.07677/) (2024)
- [ASDF: Assembly State Detection Utilizing Late Fusion by Integrating 6D Pose Estimation](https://mlsanity.com/paper/2403.16400/) (2024)
- [Active Coarse-to-Fine Segmentation of Moveable Parts from Real Images](https://mlsanity.com/paper/2303.11530/) (2023)
- [You Only Pose Once: A Minimalist's Detection Transformer for Monocular RGB Category-level 9D Multi-Object Pose Estimation](https://mlsanity.com/paper/2508.14965/) (2025)
- [TransPose: A Transformer-based 6D Object Pose Estimation Network with Depth Refinement](https://mlsanity.com/paper/2307.05561/) (2023)
- [Key Point-based Orientation Estimation of Strawberries for Robotic Fruit Picking](https://mlsanity.com/paper/2310.11333/) (2023)
- [SEMPose: A Single End-to-end Network for Multi-object Pose Estimation](https://mlsanity.com/paper/2411.14002/) (2024)
- [Vanishing Depth: Training Generalized Depth Adapters with Sinusoidal Depth Preprocessing for Pretrained RGB Encoders](https://mlsanity.com/paper/2503.19947/) (2025)
- [MVTOP: Multi-View Transformer-based Object Pose-Estimation](https://mlsanity.com/paper/2508.03243/) (2025)
- [Aleatoric Uncertainty from AI-based 6D Object Pose Predictors for Object-relative State Estimation](https://mlsanity.com/paper/2509.01583/) (2025)
- [MF-UAVPose6D: A Model-Free Monocular 6-DoF Pose Estimation Framework for Fixed-Wing UAVs](https://mlsanity.com/paper/2606.29697/) (2026)
- [TinyDETR-Pose: Towards End-to-End Real-Time Single-Stage 6DoF Object Pose Estimation with Lightweight Transformers](https://mlsanity.com/paper/2608.15297/) (2026)

## Abstract (arXiv)

Accurate 6D object pose estimation is an important task for a variety of robotic applications such as grasping or localization. It is a challenging task due to object symmetries, clutter and occlusion, but it becomes more challenging when additional information, such as depth and 3D models, is not provided. We present a transformer-based approach that takes an RGB image as input and predicts a 6D pose for each object in the image. Besides the image, our network does not require any additional information such as depth maps or 3D object models. First, the image is passed through an object detector to generate feature maps and to detect objects. Then, the feature maps are fed into a transformer with the detected bounding boxes as additional information. Afterwards, the output object queries are processed by a separate translation and rotation head. We achieve state-of-the-art results for RGB-only approaches on the challenging YCB-V dataset. We illustrate the suitability of the resulting model as pose sensor for a 6-DoF state estimation task. Code is available at https://github.com/aau-cns/poet.