mlsanityData to 7 Oct 2026

Improving 2D-3D Dense Correspondences with Diffusion Models for 6D Object Pose Estimation

arXiv 2402.06436 ↗First posted 9 Feb 2024cs.CVcited by 2

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Estimating 2D-3D correspondences between RGB images and 3D space is a fundamental problem in 6D object pose estimation. Recent pose estimators use dense correspondence maps and Point-to-Point algorithms to estimate object poses. The accuracy of pose estimation depends heavily on the quality of the dense correspondence maps and their ability to withstand occlusion, clutter, and challenging material properties. Currently, dense correspondence maps are estimated using image-to-image translation models based on GANs, Autoencoders, or direct regression models. However, recent advancements in image-to-image translation have led to diffusion models being the superior choice when evaluated on benchmarking datasets. In this study, we compare image-to-image translation networks based on GANs and diffusion models for the downstream task of 6D object pose estimation. Our results demonstrate that the diffusion-based image-to-image translation model outperforms the GAN, revealing potential for further improvements in 6D object pose estimation models.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 22 of its 28 references that are in the map. First posted on arXiv 9 Feb 2024.

Builds on

22 of 28 references.

References · 22

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  5. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 490
  6. 2018PVNet: Pixel-wise Voting Network for 6DoF Pose Estimationcited by 385
  7. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 352
  8. 2018BOP: Benchmark for 6D Object Pose Estimationcited by 254
  9. 2017BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depthcited by 225
  10. 2022BBDM: Image-to-image Translation with Brownian Bridge Diffusion Modelscited by 222
  11. 2019Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimationcited by 218
  12. 2020BOP Challenge 2020 on 6D Object Localizationcited by 192
10 more
  1. 2019DPOD: 6D Pose Object Detector and Refinercited by 189
  2. 2021GDRNPP: A Geometry-guided and Fully Learning-based Object Pose Estimatorcited by 148
  3. 2022ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimationcited by 115
  4. 2018Segmentation-driven 6D Object Pose Estimationcited by 104
  5. 2023EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimationcited by 79
  6. 2019BlenderProccited by 76
  7. 2023BOP Challenge 2022 on Detection, Segmentation and Pose Estimation of Specific Rigid Objectscited by 70
  8. 2019End-to-End Learnable Geometric Vision by Backpropagating PnP Optimizationcited by 65
  9. 2022InstaFormer: Instance-Aware Image-to-Image Translation with Transformercited by 28
  10. 2023Challenges for Monocular 6D Object Pose Estimation in Roboticscited by 21

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025BoxDreamer: Dreaming Box Corners for Generalizable Object Pose Estimationcited by 3
  2. 2025Multi-Modal 3D Mesh Reconstruction from Images and Textcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Zero123-6D: Zero-shot Novel View Synthesis for RGB Category-level 6D Pose Estimation
  2. 20236D-Diff: A Keypoint Diffusion Framework for 6D Object Pose Estimation
  3. 2024Object Pose Estimation via the Aggregation of Diffusion Features
  4. 2024Particle-based 6D Object Pose Estimation from Point Clouds using Diffusion Models
  5. 2023Confronting Ambiguity in 6D Object Pose Estimation via Score-Based Diffusion on SE(3)
  6. 2024Category Level 6D Object Pose Estimation from a Single RGB Image using Diffusion
  7. 2025RayPose: Ray Bundling Diffusion for Template Views in Unseen 6D Object Pose Estimation
  8. 2024DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimation