mlsanityData to 14 Oct 2026

CRT-6D: Fast 6D Object Pose Estimation with Cascaded Refinement Transformers

arXiv 2210.11718 ↗First posted 21 Oct 2022cs.CV cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Learning based 6D object pose estimation methods rely on computing large intermediate pose representations and/or iteratively refining an initial estimation with a slow render-compare pipeline. This paper introduces a novel method we call Cascaded Pose Refinement Transformers, or CRT-6D. We replace the commonly used dense intermediate representation with a sparse set of features sampled from the feature pyramid we call OSKFs(Object Surface Keypoint Features) where each element corresponds to an object keypoint. We employ lightweight deformable transformers and chain them together to iteratively refine proposed poses over the sampled OSKFs. We achieve inference runtimes 2x faster than the closest real-time state of the art methods while supporting up to 21 objects on a single model. We demonstrate the effectiveness of CRT-6D by performing extensive experiments on the LM-O and YCBV datasets. Compared to real-time methods, we achieve state of the art on LM-O and YCB-V, falling slightly behind methods with inference runtimes one order of magnitude higher. The source code is available at: https://github.com/PedroCastro/CRT-6D

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 43 of its 55 references that are in the map. First posted on arXiv 21 Oct 2022.

Builds on

43 of 55 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  8. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  9. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  10. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,577
  11. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  12. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 775
31 more
  1. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 495
  2. 2018PVNet: Pixel-wise Voting Network for 6DoF Pose Estimationcited by 385
  3. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 355
  4. 2020CosyPose: Consistent multi-view multi-object 6D pose estimationcited by 287
  5. 2018DeepIM: Deep Iterative Matching for 6D Pose Estimationcited by 283
  6. 2018BOP: Benchmark for 6D Object Pose Estimationcited by 254
  7. 2017SSD-6D: Making RGB-based 3D detection and 6D pose estimation great againcited by 243
  8. 2017BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depthcited by 225
  9. 2019Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimationcited by 218
  10. 2017Real-Time Seamless Single Shot 6D Object Pose Predictioncited by 210
  11. 2019DPOD: 6D Pose Object Detector and Refinercited by 189
  12. 2019Implicit 3D Orientation Learning for 6D Object Detection from RGB Imagescited by 170
  13. 2021MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimationcited by 151
  14. 2020EPOS: Estimating 6D Pose of Objects with Symmetriescited by 148
  15. 2021GDRNPP: A Geometry-guided and Fully Learning-based Object Pose Estimatorcited by 148
  16. 2021Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimationcited by 123
  17. 2022ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimationcited by 116
  18. 2015Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocolscited by 99
  19. 2021SurfEmb: Dense and Continuous Correspondence Distributions for Object Pose Estimation with Learnt Surface Embeddingscited by 95
  20. 2019Single-Stage 6D Object Pose Estimationcited by 88
  21. 2021SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimationcited by 88
  22. 2022OSOP: A Multi-Stage One Shot Object Pose Estimation Frameworkcited by 81
  23. 2018Making Deep Heatmaps Robust to Partial Occlusions for 3D Object Pose Estimationcited by 74
  24. 2019End-to-End Learnable Geometric Vision by Backpropagating PnP Optimizationcited by 65
  25. 2021RePOSE: Fast 6D Object Pose Refinement via Deep Texture Renderingcited by 63
  26. 2022What to look at and where: Semantic and Spatial Refined Transformer for detecting human-object interactionscited by 35
  27. 2022Towards Accurate Facial Landmark Detection via Cascaded Transformerscited by 31
  28. 20216D-ViT: Category-Level 6D Object Pose Estimation via Transformer-based Instance Representation Learningcited by 20
  29. 2022YOLOPose: Transformer-based Multi-Object 6D Pose Estimation using Keypoint Regressioncited by 17
  30. 2022SC6D: Symmetry-agnostic and Correspondence-free 6D Object Pose Estimationcited by 17
  31. 2019Accurate 6D Object Pose Estimation by Pose Conditioned Mesh Reconstructioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 6; 2025: 2; 2026: 3.

15 papers.

Sort:

By area:

  1. 2024HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interactioncited by 29
  2. 20236D-Diff: A Keypoint Diffusion Framework for 6D Object Pose Estimationcited by 24
  3. 2023CheckerPose: Progressive Dense Keypoint Localization for Object Pose Estimation with Graph Neural Networkcited by 15
  4. 2023PoseMatcher: One-shot 6D Object Pose Estimation by Deep Feature Matchingcited by 12
  5. 2024MRC-Net: 6-DoF Pose Estimation with MultiScale Residual Correlationcited by 11
  6. 2024GeoReF: Geometric Alignment Across Shape Variation for Category-level Object Pose Refinementcited by 8
  7. 2023PViT-6D: Overclocking Vision Transformers for 6D Pose Estimation with Confidence-Level Prediction and Pose Tokenscited by 1
  8. 2024Resolving Symmetry Ambiguity in Correspondence-based Methods for Instance-level Object Pose Estimationcited by 1
  9. 2024Generalizable Single-view Object Pose Estimation by Two-side Generating and Matchingcited by 1
  10. 2025Joint Learning of Pose Regression and Denoising Diffusion with Score Scaling Sampling for Category-level 6D Pose Estimationcited by 1
  11. 2026Towards Symmetry-sensitive Pose Estimation: A Rotation Representation for Symmetric Object Classescited by 1
  12. 2024Sparse Color-Code Net: Real-Time RGB-Based 6D Object Pose Estimation on Edge Devicescited by 0
3 more
  1. 2025RAG-6DPose: Retrieval-Augmented 6D Pose Estimation via Leveraging CAD as Knowledge Basecited by 0
  2. 2026DecomPose: Disentangling Cross-Category Optimization Contention for Category-Level 6D Object Pose Estimationcited by 0
  3. 2026ComPose: When to Trust Hands for Object Pose Trackingcited by 0