# RYOPO: Bringing End-to-End Category-Level Object Pose Estimation into Real Time

arXiv: https://arxiv.org/abs/2610.03013 · HTML: https://mlsanity.com/paper/2610.03013/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 38 of its 41 references that are in the map.
- First posted on arXiv 2 Oct 2026.

## Builds on (most cited first, 30 of 38)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries](https://mlsanity.com/paper/2110.06922/) (2021)
- [PETR: Position Embedding Transformation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2203.05625/) (2022)
- [DETRs Beat YOLOs on Real-time Object Detection](https://mlsanity.com/paper/2304.08069/) (2023)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion](https://mlsanity.com/paper/1901.04780/) (2019)
- [Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation](https://mlsanity.com/paper/2206.02777/) (2022)
- [MVX-Net: Multimodal VoxelNet for 3D Object Detection](https://mlsanity.com/paper/1904.01649/) (2019)
- [Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation](https://mlsanity.com/paper/2007.08454/) (2020)
- [Multimodal Token Fusion for Vision Transformers](https://mlsanity.com/paper/2204.08721/) (2022)
- [RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer](https://mlsanity.com/paper/2407.17140/) (2024)
- [Category-Level 6D Object Pose and Size Estimation using Self-Supervised Deep Prior Deformation Networks](https://mlsanity.com/paper/2207.05444/) (2022)
- [ShAPO: Implicit Representations for Multi-Object Shape, Appearance, and Pose Optimization](https://mlsanity.com/paper/2207.13691/) (2022)
- [CenterSnap: Single-Shot Multi-Object 3D Shape Reconstruction and Categorical 6D Pose and Size Estimation](https://mlsanity.com/paper/2203.01929/) (2022)
- [Cross Modal Transformer: Towards Fast and Robust 3D Object Detection](https://mlsanity.com/paper/2301.01283/) (2023)
- [HouseCat6D -- A Large-Scale Multi-Modal Category Level 6D Object Perception Dataset with Household Objects in Realistic Scenarios](https://mlsanity.com/paper/2212.10428/) (2022)
- [SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimation](https://mlsanity.com/paper/2311.11125/) (2023)
- [HS-Pose: Hybrid Scope Feature Extraction for Category-level Object Pose Estimation](https://mlsanity.com/paper/2303.15743/) (2023)
- [Instance-Adaptive and Geometric-Aware Keypoint Learning for Category-Level 6D Object Pose Estimation](https://mlsanity.com/paper/2403.19527/) (2024)
- [VI-Net: Boosting Category-level 6D Object Pose Estimation via Learning Decoupled Rotations on the Spherical Representations](https://mlsanity.com/paper/2308.09916/) (2023)
- [V-DETR: DETR with Vertex Relative Position Encoding for 3D Object Detection](https://mlsanity.com/paper/2308.04409/) (2023)
- [RT-DETRv3: Real-time End-to-End Object Detection with Hierarchical Dense Positive Supervision](https://mlsanity.com/paper/2409.08475/) (2024)
- [CATRE: Iterative Point Clouds Alignment for Category-level Object Pose Refinement](https://mlsanity.com/paper/2207.08082/) (2022)
- [Uni3DETR: Unified 3D Detection Transformer](https://mlsanity.com/paper/2310.05699/) (2023)
- [Bridged Transformer for Vision and Point Cloud 3D Object Detection](https://mlsanity.com/paper/2210.01391/) (2022)

## Abstract (arXiv)

Category-level object pose estimation predicts the rotation, translation, and metric size of unseen instances within known categories. Many accurate RGB-D methods rely on external instance segmentation and crop-based pose estimation, introducing separate stages and object-dependent processing costs that hinder real-time inference. To bring accurate pose estimation into real time, we present ours, an end-to-end trainable query-based RGB-D set predictor. It jointly detects and segments objects and estimates their mbox9-DoF poses without explicit CAD-derived shape priors or a separately trained instance segmentor. Shared image and scene encoding avoids repeated per-object crop encoding. A query-conditioned geometry pathway associates observed 3D points and RGB features with object queries and incorporates shared scene context. Object-centric refinement uses the resulting point descriptors to update an explicit pose state through pose-conditioned cross-attention and recurrent residual corrections. On NOCS, ours substantially improves on published RGB-D joint detection and pose estimation results. It achieves competitive performance compared with two-stage methods under all-object evaluation on REAL275 and HouseCat6D, while enabling real-time full-frame pose estimation at 31.8 FPS on an RTX~A6000. Project page: https://yopo-series.github.io/RYOPO-project-page/.