mlsanityData to 7 Oct 2026

Cameras as Rays: Pose Estimation via Ray Diffusion

arXiv 2402.14817 ↗First posted 22 Feb 2024cs.CV cs.LGcited by 129

Not on the map. Why?

Cited by 26 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Estimating camera poses is a fundamental task for 3D reconstruction and remains challenging given sparsely sampled views (<10). In contrast to existing approaches that pursue top-down prediction of global parametrizations of camera extrinsics, we propose a distributed representation of camera pose that treats a camera as a bundle of rays. This representation allows for a tight coupling with spatial image features improving pose precision. We observe that this representation is naturally suited for set-level transformers and develop a regression-based approach that maps image patches to corresponding rays. To capture the inherent uncertainties in sparse-view pose inference, we adapt this approach to learn a denoising diffusion model which allows us to sample plausible modes while improving performance. Our proposed methods, both regression- and diffusion-based, demonstrate state-of-the-art performance on camera pose estimation on CO3D while generalizing to unseen object categories and in-the-wild captures.

Key facts. Cited by 129 papers in the map (data through 7 Oct 2026). Builds on 36 of its 55 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

36 of 55 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2022Scalable Diffusion Models with Transformerscited by 5,931
  6. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  7. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  8. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,313
  9. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 983
  10. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 862
  11. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 801
  12. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
24 more
  1. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  2. 2018From Coarse to Fine: Robust Hierarchical Localization at Large Scalecited by 556
  3. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 330
  4. 2022Novel View Synthesis with Diffusion Modelscited by 299
  5. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  6. 2021Pixel-Perfect Structure-from-Motion with Featuremetric Refinementcited by 199
  7. 2022SPARF: Neural Radiance Fields from Sparse and Noisy Posescited by 195
  8. 2022SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Viewscited by 175
  9. 2018BA-Net: Dense Bundle Adjustment Networkcited by 173
  10. 2023PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Predictioncited by 165
  11. 2023PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustmentcited by 146
  12. 2021NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wildcited by 136
  13. 2022RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wildcited by 102
  14. 2023RelPose++: Recovering 6D Poses from Sparse-view Observationscited by 87
  15. 2022SparsePose: Sparse-View Camera Pose Regression and Refinementcited by 53
  16. 2022Few-View Object Reconstruction with Unknown Categories and Camera Posescited by 53
  17. 2021Wide-Baseline Relative Camera Pose Estimation with Directional Learningcited by 49
  18. 2020RANSAC-Flow: generic two-stage image alignmentcited by 48
  19. 2019Why Having 10,000 Parameters in Your Camera Model is Better Than Twelvecited by 43
  20. 2021Extreme Rotation Estimation using Dense Correlation Volumescited by 42
  21. 2022The 8-Point Algorithm as an Inductive Bias for Relative Pose Prediction by ViTscited by 38
  22. 2021Differentiable Stereopsis: Meshes from multiple views using differentiable renderingcited by 26
  23. 2020Neural Ray Surfaces for Self-Supervised Learning of Depth and Ego-motioncited by 20
  24. 2023Sparse 3D Reconstruction via Object-Centric Ray Samplingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 35; 2025: 60; 2026: 34.

129 papers.

Sort:

By area:

  1. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  2. 2024Grounding Image Matching in 3D with MASt3Rcited by 960
  3. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  4. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 356
  5. 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 295
  6. 2024Global Structure-from-Motion Revisitedcited by 215
  7. 2025FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Viewscited by 176
  8. 2024MASt3R-SfM: a Fully-Integrated Solution for Unconstrained Structure-from-Motioncited by 161
  9. 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 139
  10. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 113
  11. 2025Cameras as Relative Positional Encodingcited by 94
  12. 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 84
38 more
  1. 2024Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localizationcited by 81
  2. 2025RayZer: A Self-supervised Large View Synthesis Modelcited by 74
  3. 2024Scene Coordinate Reconstruction: Posing of Image Collections via Incremental Learning of a Relocalizercited by 73
  4. 2025Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priorscited by 69
  5. 2025Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstructioncited by 66
  6. 2024Video-Guided Foley Sound Generation with Multimodal Controlscited by 49
  7. 2025Dens3R: A Foundation Model for 3D Geometry Predictioncited by 44
  8. 2024One Diffusion to Generate Them Allcited by 41
  9. 2024FreeSplatter: Pose-free Gaussian Splatting for Sparse-view 3D Reconstructioncited by 40
  10. 2025Matrix3D: Large Photogrammetry Model All-in-Onecited by 39
  11. 2025AerialMegaDepth: Learning Aerial-Ground Reconstruction and View Synthesiscited by 39
  12. 2025DeepVerse: 4D Autoregressive Video Generation as a World Modelcited by 36
  13. 2024SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting Synthesiscited by 34
  14. 2024Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generationcited by 33
  15. 2025OmniNWM: Omniscient Driving Navigation World Modelscited by 32
  16. 2024Robust Incremental Structure-from-Motion with Hybrid Featurescited by 31
  17. 2025FlowR: Flowing from Sparse to Dense 3D Reconstructionscited by 28
  18. 2025PlayerOne: Egocentric World Simulatorcited by 20
  19. 2025Particulate: Feed-Forward 3D Object Articulationcited by 20
  20. 2025Thinking with Camera: A Unified Multimodal Model for Camera-Centric Understanding and Generationcited by 19
  21. 2024Images that Sound: Composing Images and Sounds on a Single Canvascited by 17
  22. 2024LoRA3D: Low-Rank Self-Calibration of 3D Geometric Foundation Modelscited by 17
  23. 2024AKiRa: Augmentation Kit on Rays for optical video generationcited by 17
  24. 2025Diffuman4D: 4D Consistent Human View Synthesis from Sparse-View Videos with Spatio-Temporal Diffusion Modelscited by 17
  25. 2025Seeing to Act, Prompting to Specify: A Bayesian Factorization of Vision Language Action Policycited by 17
  26. 2024PuzzleAvatar: Assembling 3D Avatars from Personal Albumscited by 16
  27. 2025SpatialCrafter: Unleashing the Imagination of Video Diffusion Models for Scene Reconstruction from Limited Observationscited by 16
  28. 2025SPATIALGEN: Layout-guided 3D Indoor Scene Generationcited by 14
  29. 2025Do You Know Where Your Camera Is? View-Invariant Policy Learning with Camera Conditioningcited by 14
  30. 2024Generative Multiview Relighting for 3D Reconstruction under Extreme Illumination Variationcited by 13
  31. 2025Rectified Point Flow: Generic Point Cloud Pose Estimationcited by 13
  32. 2024NVComposer: Boosting Generative Novel View Synthesis with Multiple Sparse and Unposed Imagescited by 12
  33. 2025L4P: Towards Unified Low-Level 4D Vision Perceptioncited by 12
  34. 2025VideoRFSplat: Direct Scene-Level Text-to-3D Gaussian Splatting Generation with Flexible Pose and Multi-View Joint Modelingcited by 12
  35. 2026Video Generation Models are General-Purpose Vision Learnerscited by 12
  36. 2025Rig3R: Rig-Aware Conditioning for Learned 3D Reconstructioncited by 11
  37. 2025Can Video Diffusion Model Reconstruct 4D Geometry?cited by 10
  38. 2025ReDirector: Creating Any-Length Video Retakes with Rotary Camera Encodingcited by 10

50 most cited of 129 All 129 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023ID-Pose: Sparse-view Camera Pose Estimation by Inverting Diffusion Models
  2. 2024SpaRP: Fast 3D Object Reconstruction and Pose Estimation from Sparse Views
  3. 2024Object Pose Estimation via the Aggregation of Diffusion Features
  4. 2023iFusion: Inverting Diffusion for Pose-Free Reconstruction from Sparse Views
  5. 2025DiffusionSfM: Predicting Structure and Motion via Ray Origin and Endpoint Diffusion
  6. 2022DiffPose: Toward More Reliable 3D Pose Estimation
  7. 2024D3PRefiner: A Diffusion-based Denoise Method for 3D Human Pose Refinement
  8. 2022DiffPose: Multi-hypothesis Human Pose Estimation using Diffusion models