mlsanityData to 14 Oct 2026

Zero123-6D: Zero-shot Novel View Synthesis for RGB Category-level 6D Pose Estimation

arXiv 2403.14279 ↗First posted 21 Mar 2024cs.CVcited by 10

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Estimating the pose of objects through vision is essential to make robotic platforms interact with the environment. Yet, it presents many challenges, often related to the lack of flexibility and generalizability of state-of-the-art solutions. Diffusion models are a cutting-edge neural architecture transforming 2D and 3D computer vision, outlining remarkable performances in zero-shot novel-view synthesis. Such a use case is particularly intriguing for reconstructing 3D objects. However, localizing objects in unstructured environments is rather unexplored. To this end, this work presents Zero123-6D, the first work to demonstrate the utility of Diffusion Model-based novel-view-synthesizers in enhancing RGB 6D pose estimation at category-level, by integrating them with feature extraction techniques. Novel View Synthesis allows to obtain a coarse pose that is refined through an online optimization method introduced in this work to deal with intra-category geometric differences. In such a way, the outlined method shows reduction in data requirements, removal of the necessity of depth information in zero-shot category-level 6D pose estimation task, and increased performance, quantitatively demonstrated through experiments on the CO3D dataset.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 46 of its 55 references that are in the map. First posted on arXiv 21 Mar 2024.

Builds on

46 of 55 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024EscherNet: A Generative Model for Scalable View Synthesis6 Feb
  2. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion14 Nov
  3. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
  4. 2023Consistent123: Improve Consistency for One Image to 3D Object Synthesis12 Oct
  5. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun
  6. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  7. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov

Other references · 39

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  7. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  8. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  9. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  10. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  11. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 984
  12. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 775
27 more
  1. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
  2. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 495
  3. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 429
  4. 2018BOP: Benchmark for 6D Object Pose Estimationcited by 254
  5. 2022OnePose: One-Shot Object Pose Estimation without CAD Modelscited by 157
  6. 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
  7. 2021GDRNPP: A Geometry-guided and Fully Learning-based Object Pose Estimatorcited by 148
  8. 2021FS-Net: Fast Shape-based Network for Category-Level 6D Object Pose Estimation with Decoupled Rotation Mechanismcited by 121
  9. 2022ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimationcited by 116
  10. 2023OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Modelscited by 115
  11. 2023FoundPose: Unseen Object Pose Estimation with Foundation Featurescited by 109
  12. 2021DualPoseNet: Category-level 6D Object Pose and Size Estimation Using Dual Pose Network with Refined Learning of Pose Consistencycited by 103
  13. 2022OSOP: A Multi-Stage One Shot Object Pose Estimation Frameworkcited by 81
  14. 2022Category-Level 6D Object Pose and Size Estimation using Self-Supervised Deep Prior Deformation Networkscited by 74
  15. 2023BOP Challenge 2022 on Detection, Segmentation and Pose Estimation of Specific Rigid Objectscited by 70
  16. 2022NeRF-Pose: A First-Reconstruct-Then-Regress Approach for Weakly-supervised 6D Object Pose Estimationcited by 58
  17. 2022Zero-Shot Category-Level Object Pose Estimationcited by 57
  18. 2022HouseCat6D -- A Large-Scale Multi-Modal Category Level 6D Object Perception Dataset with Household Objects in Realistic Scenarioscited by 53
  19. 2023SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimationcited by 53
  20. 2023ZS6D: Zero-shot 6D Object Pose Estimation using Vision Transformerscited by 45
  21. 2023VI-Net: Boosting Category-level 6D Object Pose Estimation via Learning Decoupled Rotations on the Spherical Representationscited by 41
  22. 2023NOPE: Novel Object Pose Estimation from a Single Imagecited by 38
  23. 2022Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimationcited by 34
  24. 2023POPE: 6-DoF Promptable Pose Estimation of Any Object, in Any Scene, with One Referencecited by 30
  25. 2022TexPose: Neural Texture Learning for Self-Supervised 6D Object Pose Estimationcited by 23
  26. 2023You Only Look at One: Category-Level Object Representations for Pose Estimation From a Single Examplecited by 8
  27. 2023S2P3: Self-Supervised Polarimetric Pose Predictioncited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 4; 2026: 1.

10 papers.

  1. 2024Deep Learning-Based Object Pose Estimation: A Comprehensive Surveycited by 40
  2. 2025Accurate and efficient zero-shot 6D pose estimation with frozen foundation modelscited by 13
  3. 2024Diffusion Features for Zero-Shot 6DoF Object Pose Estimationcited by 4
  4. 2025HIPPo: Harnessing Image-to-3D Priors for Model-free Zero-shot 6D Pose Estimationcited by 4
  5. 2024Enhancing Single Image to 3D Generation using Gaussian Splatting and Hybrid Diffusion Priorscited by 2
  6. 2024Segment Anything in Light Fields for Real-Time Applications via Constrained Promptingcited by 2
  7. 2025SingRef6D: Monocular Novel Object Pose Estimation with a Single RGB Referencecited by 1
  8. 2024SABER-6D: Shape Representation Based Implicit Object Pose Estimationcited by 0
  9. 2025Color-Pair Guided Robust Zero-Shot 6D Pose Estimation and Tracking of Cluttered Objects on Edge Devicescited by 0
  10. 2026Pose Anything Anywhere:Model-free Object Poses from Arbitrary Referencescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025UnPose: Uncertainty-Guided Diffusion Priors for Zero-Shot Pose Estimation
  2. 2024Category Level 6D Object Pose Estimation from a Single RGB Image using Diffusion
  3. 2024Improving 2D-3D Dense Correspondences with Diffusion Models for 6D Object Pose Estimation
  4. 2023Confronting Ambiguity in 6D Object Pose Estimation via Score-Based Diffusion on SE(3)
  5. 2025Beyond 'Templates': Category-Agnostic Object Pose, Size, and Shape Estimation from a Single View
  6. 2025Diff9D: Diffusion-Based Domain-Generalized Category-Level 9-DoF Object Pose Estimation
  7. 2023Challenges for Monocular 6D Object Pose Estimation in Robotics
  8. 2025AxisPose: Model-Free Matching-Free Single-Shot 6D Object Pose Estimation via Axis Generation