mlsanityData to 7 Oct 2026

Extreme Two-View Geometry From Object Poses with Diffusion Models

arXiv 2402.02800 ↗First posted 5 Feb 2024cs.CVcited by 4

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Human has an incredible ability to effortlessly perceive the viewpoint difference between two images containing the same object, even when the viewpoint change is astonishingly vast with no co-visible regions in the images. This remarkable skill, however, has proven to be a challenge for existing camera pose estimation methods, which often fail when faced with large viewpoint differences due to the lack of overlapping local features for matching. In this paper, we aim to effectively harness the power of object priors to accurately determine two-view geometry in the face of extreme viewpoint changes. In our method, we first mathematically transform the relative camera pose estimation problem to an object pose estimation problem. Then, to estimate the object pose, we utilize the object priors learned from a diffusion model Zero123 to synthesize novel-view images of the object. The novel-view images are matched to determine the object pose and thus the two-view camera pose. In experiments, our method has demonstrated extraordinary robustness and resilience to large viewpoint changes, consistently estimating two-view poses with exceptional generalization ability across both synthetic and real-world datasets. Code will be available at https://github.com/scy639/Extreme-Two-View-Geometry-From-Object-Poses-with-Diffusion-Models.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 66 of its 77 references that are in the map. First posted on arXiv 5 Feb 2024.

Builds on

66 of 77 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
  2. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
  3. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  4. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun
  5. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar

Other references · 61

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023Segment Anythingcited by 8,158
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  6. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  7. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  8. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  9. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  10. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 983
  11. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
  12. 2017End-to-end Recovery of Human Shape and Posecited by 883
38 more
  1. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 862
  2. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 801
  3. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
  4. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  5. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  6. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  7. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  8. 2020INeRF: Inverting Neural Radiance Fields for Pose Estimationcited by 381
  9. 2018Learning 3D Human Dynamics from Videocited by 315
  10. 2018DeepIM: Deep Iterative Matching for 6D Pose Estimationcited by 283
  11. 2020D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometrycited by 206
  12. 2017DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networkscited by 173
  13. 2023PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Predictioncited by 165
  14. 2022OnePose: One-Shot Object Pose Estimation without CAD Modelscited by 155
  15. 2019Learning Two-View Correspondences and Geometry Using Order-Aware Networkcited by 148
  16. 2023BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objectscited by 144
  17. 2017VNect: Real-time 3D Human Pose Estimation with a Single RGB Cameracited by 129
  18. 2022Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Imagescited by 120
  19. 2023OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Modelscited by 114
  20. 2023GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondencecited by 113
  21. 2022Map-free Visual Relocalization: Metric Pose Relative to a Single Imagecited by 112
  22. 2022RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wildcited by 102
  23. 2019LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimationcited by 88
  24. 2021BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Modelscited by 87
  25. 2023RelPose++: Recovering 6D Poses from Sparse-view Observationscited by 87
  26. 2016Universal Correspondence Networkcited by 82
  27. 2022FS6D: Few-Shot 6D Pose Estimation of Novel Objectscited by 81
  28. 2022OSOP: A Multi-Stage One Shot Object Pose Estimation Frameworkcited by 81
  29. 2022ShAPO: Implicit Representations for Multi-Object Shape, Appearance, and Pose Optimizationcited by 70
  30. 2022CenterSnap: Single-Shot Multi-Object 3D Shape Reconstruction and Categorical 6D Pose and Size Estimationcited by 69
  31. 2022NeRF-Pose: A First-Reconstruct-Then-Regress Approach for Weakly-supervised 6D Object Pose Estimationcited by 57
  32. 2022Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusionscited by 54
  33. 2022SparsePose: Sparse-View Camera Pose Regression and Refinementcited by 53
  34. 2022Few-View Object Reconstruction with Unknown Categories and Camera Posescited by 53
  35. 2023NAVI: Category-Agnostic Image Collections with High-Quality 3D Shape and Pose Annotationscited by 53
  36. 2017Relative Camera Pose Estimation Using Convolutional Neural Networkscited by 47
  37. 2022OVE6D: Object Viewpoint Encoding for Depth-based 6D Object Pose Estimationcited by 43
  38. 2019Pose from Shape: Deep Pose Estimation for Arbitrary 3D Objectscited by 39

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2024Sparse-view Pose Estimation and Reconstruction via Analysis by Generative Synthesiscited by 9
  2. 2025BoxDreamer: Dreaming Box Corners for Generalizable Object Pose Estimationcited by 3
  3. 2024Generalizable Single-view Object Pose Estimation by Two-side Generating and Matchingcited by 1
  4. 2026CGGT: Curve-Grounded Geometry Transformer for 3D Parametric Curve Reconstructioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Object Pose Estimation via the Aggregation of Diffusion Features
  2. 2025SDPose: Exploiting Diffusion Priors for Out-of-Domain and Robust Pose Estimation
  3. 2026Object Pose Transformer: Unifying Unseen Object Pose Estimation
  4. 2023ID-Pose: Sparse-view Camera Pose Estimation by Inverting Diffusion Models
  5. 2023iFusion: Inverting Diffusion for Pose-Free Reconstruction from Sparse Views
  6. 2026Flexible Geometric Guidance for Probabilistic Human Pose Estimation with Diffusion Models
  7. 2024Di2Pose: Discrete Diffusion Model for Occluded 3D Human Pose Estimation
  8. 2024DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimation