mlsanityData to 7 Oct 2026

Zero-Shot Category-Level Object Pose Estimation

arXiv 2204.03635 ↗First posted 7 Apr 2022cs.CV cs.ROcited by 57

Not on the map. Why?

Abstract (arXiv)

Object pose estimation is an important component of most vision pipelines for embodied agents, as well as in 3D vision more generally. In this paper we tackle the problem of estimating the pose of novel object categories in a zero-shot manner. This extends much of the existing literature by removing the need for pose-labelled datasets or category-specific CAD models for training or inference. Specifically, we make the following contributions. First, we formalise the zero-shot, category-level pose estimation problem and frame it in a way that is most applicable to real-world embodied agents. Secondly, we propose a novel method based on semantic correspondences from a self-supervised vision transformer to solve the pose estimation problem. We further re-purpose the recent CO3D dataset to present a controlled and realistic test setting. Finally, we demonstrate that all baselines for our proposed task perform poorly, and show that our method provides a six-fold improvement in average rotation accuracy at 30 degrees. Our code is available at https://github.com/applied-ai-lab/zero-shot-pose.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). Builds on 40 of its 56 references that are in the map. First posted on arXiv 7 Apr 2022.

Builds on

40 of 56 references.

References · 40

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  3. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  4. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  5. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  6. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  7. 2018Open3D: A Modern Library for 3D Data Processingcited by 717
  8. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  9. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 490
  10. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  11. 2022Generalized Category Discoverycited by 209
  12. 2021Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulationcited by 204
28 more
  1. 2020Shape Prior Deformation for Categorical 6D Object Pose and Size Estimationcited by 158
  2. 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
  3. 2019kPAM: KeyPoint Affordances for Category-Level Robotic Manipulationcited by 152
  4. 2020Few-Shot Object Detection and Viewpoint Estimation for Objects in the Wildcited by 142
  5. 2018Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulationcited by 140
  6. 2020Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimationcited by 123
  7. 2021FS-Net: Fast Shape-based Network for Category-Level 6D Object Pose Estimation with Decoupled Rotation Mechanismcited by 121
  8. 2021Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Losscited by 116
  9. 2019Self-supervised 6D Object Pose Estimation for Robot Manipulationcited by 100
  10. 2020Category Level Object Pose Estimation via Neural Analysis-by-Synthesiscited by 100
  11. 20176-DoF Object Pose from Semantic Keypointscited by 89
  12. 2016A Large Dataset of Object Scanscited by 64
  13. 2019SFNet: Learning Object-aware Semantic Correspondencecited by 60
  14. 2021Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimationcited by 45
  15. 2018StarMap for Category-Agnostic Keypoint and Viewpoint Estimationcited by 42
  16. 2021Single-Stage Keypoint-Based Category-Level Object Pose Estimation from an RGB Imagecited by 42
  17. 2021NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimationcited by 41
  18. 2019Pose from Shape: Deep Pose Estimation for Arbitrary 3D Objectscited by 39
  19. 20183D Pose Estimation and 3D Model Retrieval for Objects in the Wildcited by 31
  20. 2022ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapescited by 31
  21. 2018Category-level 6D Object Pose Recovery in Depth Imagescited by 21
  22. 2021Semantically Grounded Object Matching for Robust Robotic Scene Rearrangementcited by 21
  23. 2021PoseContrast: Class-Agnostic Object Viewpoint Estimation in the Wild with Pose-Aware Contrastive Learningcited by 19
  24. 2021Optimal Pose and Shape Estimation for Category-level 3D Object Perceptioncited by 15
  25. 2018Object Pose Estimation from Monocular Image using Multi-View Keypoint Correspondencecited by 8
  26. 2020Novel Object Viewpoint Estimation through Reconstruction Alignmentcited by 8
  27. 2019Few-Shot Viewpoint Estimationcited by 6
  28. 2015Inferring 3D Object Pose in RGB-D Imagescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 17; 2024: 19; 2025: 8; 2026: 7.

57 papers.

Sort:

By area:

  1. 2023Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondencecited by 211
  2. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 175
  3. 2024HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videoscited by 122
  4. 2022Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Imagescited by 120
  5. 2023SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimationcited by 115
  6. 2023FoundPose: Unseen Object Pose Estimation with Foundation Featurescited by 109
  7. 2022ShAPO: Implicit Representations for Multi-Object Shape, Appearance, and Pose Optimizationcited by 70
  8. 2023FreeZe: Training-free zero-shot 6D pose estimation with geometric and vision foundation modelscited by 56
  9. 2023SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimationcited by 52
  10. 2023ZS6D: Zero-shot 6D Object Pose Estimation using Vision Transformerscited by 45
  11. 2024Deep Learning-Based Object Pose Estimation: A Comprehensive Surveycited by 39
  12. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 34
38 more
  1. 2023A General Protocol to Probe Large Vision Models for 3D Physical Understandingcited by 32
  2. 2023POPE: 6-DoF Promptable Pose Estimation of Any Object, in Any Scene, with One Referencecited by 30
  3. 2022Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wildcited by 22
  4. 2024OmniNOCS: A unified NOCS dataset and model for 3D lifting of 2D objectscited by 22
  5. 2024Open Vocabulary Monocular 3D Object Detectioncited by 22
  6. 20223D Vision with Transformers: A Surveycited by 21
  7. 2023Challenges for Monocular 6D Object Pose Estimation in Roboticscited by 21
  8. 2023Learning to Estimate 6DoF Pose from Limited Data: A Few-Shot, Generalizable Approach using RGB Imagescited by 18
  9. 2023FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objectscited by 18
  10. 2024The 3D-PC: a benchmark for visual perspective taking in humans and machinescited by 16
  11. 2025GCE-Pose: Global Context Enhancement for Category-level Object Pose Estimationcited by 16
  12. 2023iComMa: Inverting 3D Gaussian Splatting for Camera Pose Estimation via Comparing and Matchingcited by 12
  13. 2023NOVUM: Neural Object Volumes for Robust Object Classificationcited by 11
  14. 20233D-Aware Hypothesis & Verification for Generalizable Relative Object Pose Estimationcited by 11
  15. 2024Zero123-6D: Zero-shot Novel View Synthesis for RGB Category-level 6D Pose Estimationcited by 10
  16. 2024Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modelingcited by 10
  17. 2024SDFit: 3D Object Pose and Shape by Fitting a Morphable SDF to a Single Imagecited by 9
  18. 2025Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Spacecited by 9
  19. 2023You Only Look at One: Category-Level Object Representations for Pose Estimation From a Single Examplecited by 8
  20. 2024Source-Free and Image-Only Unsupervised Domain Adaptation for Category Level Object Pose Estimationcited by 7
  21. 2025GECO: Geometrically Consistent Embedding with Lightspeed Inferencecited by 6
  22. 2024Unsupervised Learning of Category-Level 3D Pose from Object-Centric Videoscited by 5
  23. 2024SRPose: Two-view Relative Pose Estimation with Sparse Keypointscited by 5
  24. 2025GIVEPose: Gradual Intra-class Variation Elimination for RGB-based Category-Level Object Pose Estimationcited by 5
  25. 2024Diffusion Features for Zero-Shot 6DoF Object Pose Estimationcited by 4
  26. 2025Structure-Aware Correspondence Learning for Relative Pose Estimationcited by 3
  27. 2026Scaling Single Human Demonstrations for Imitation Learning using Generative Foundational Modelscited by 3
  28. 2022ObPose: Leveraging Pose for Object-Centric Scene Inference and Generation in 3Dcited by 2
  29. 2022ViewNeRF: Unsupervised Viewpoint Estimation Using Category-Level Neural Radiance Fieldscited by 2
  30. 2023Open Challenges for Monocular Single-shot 6D Object Pose Estimationcited by 2
  31. 2023Grasp Transfer based on Self-Aligning Implicit Representations of Local Surfacescited by 2
  32. 2024GeoBiked: A Dataset with Geometric Features and Automated Labeling Techniques to Enable Deep Generative Models in Engineering Designcited by 2
  33. 2025ViT-VS: On the Applicability of Pretrained Vision Transformer Features for Generalizable Visual Servoingcited by 2
  34. 2025Reconstructing Objects along Hand Interaction Timelines in Egocentric Videocited by 2
  35. 20243D Congealing: 3D-Aware Image Alignment in the Wildcited by 1
  36. 2024Generalizable Single-view Object Pose Estimation by Two-side Generating and Matchingcited by 1
  37. 2025Monocular Visual 8D Pose Estimation for Articulated Bicycles and Cyclistscited by 1
  38. 2026COG: Confidence-aware Optimal Geometric Correspondence for Unsupervised Single-reference Novel Object Pose Estimationcited by 1

50 most cited of 57 All 57 (JSON)