mlsanityData to 7 Oct 2026

Unifying Correspondence, Pose and NeRF for Pose-Free Novel View Synthesis from Stereo Pairs

arXiv 2312.07246 ↗First posted 12 Dec 2023cs.CVcited by 37

Not on the map. Why?

Abstract (arXiv)

This work delves into the task of pose-free novel view synthesis from stereo pairs, a challenging and pioneering task in 3D vision. Our innovative framework, unlike any before, seamlessly integrates 2D correspondence matching, camera pose estimation, and NeRF rendering, fostering a synergistic enhancement of these tasks. We achieve this through designing an architecture that utilizes a shared representation, which serves as a foundation for enhanced 3D geometry understanding. Capitalizing on the inherent interplay between the tasks, our unified framework is trained end-to-end with the proposed training strategy to improve overall model accuracy. Through extensive evaluations across diverse indoor and outdoor scenes from two real-world datasets, we demonstrate that our approach achieves substantial improvement over previous methodologies, especially in scenarios characterized by extreme viewpoint changes and the absence of accurate camera poses.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 50 of its 73 references that are in the map. First posted on arXiv 12 Dec 2023.

Builds on

50 of 73 references.

References · 50

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  4. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  5. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  6. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  7. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,313
  8. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  9. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
  10. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  11. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  12. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 641
38 more
  1. 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 586
  2. 2021GMFlow: Learning Optical Flow via Global Matchingcited by 324
  3. 2016LIFT: Learned Invariant Feature Transformcited by 305
  4. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 273
  5. 2016DSAC - Differentiable RANSAC for Camera Localizationcited by 264
  6. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
  7. 2022NoPe-NeRF: Optimising Neural Radiance Field with No Pose Priorcited by 235
  8. 2020Deep Global Registrationcited by 228
  9. 2023CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentationcited by 215
  10. 2021Self-Calibrating Neural Radiance Fieldscited by 204
  11. 2017UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Losscited by 202
  12. 2021GeoNeRF: Generalizing NeRF with Geometry Priorscited by 195
  13. 2022SPARF: Neural Radiance Fields from Sparse and Noisy Posescited by 195
  14. 2017Learning to Find Good Correspondencescited by 176
  15. 2019MAGSAC++, a fast, reliable and accurate robust estimatorcited by 160
  16. 2018MAGSAC: marginalizing sample consensuscited by 122
  17. 2022Generalizable Patch-Based Neural Renderingcited by 121
  18. 2022Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentationcited by 114
  19. 2022Map-free Visual Relocalization: Metric Pose Relative to a Single Imagecited by 112
  20. 2023Learning to Render Novel Views from Wide-Baseline Stereo Pairscited by 103
  21. 2022RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wildcited by 102
  22. 2021PDC-Net+: Enhanced Probabilistic Dense Correspondence Networkcited by 80
  23. 2017Camera Relocalization by Computing Pairwise Relative Poses Using Convolutional Neural Networkcited by 79
  24. 2021CATs: Cost Aggregation Transformers for Visual Correspondencecited by 77
  25. 2023DBARF: Deep Bundle-Adjusting Generalizable Neural Radiance Fieldscited by 60
  26. 2022CATs++: Boosting Cost Aggregation with Convolutions and Transformerscited by 53
  27. 2021Wide-Baseline Relative Camera Pose Estimation with Directional Learningcited by 49
  28. 2017Relative Camera Pose Estimation Using Convolutional Neural Networkscited by 47
  29. 2023Explicit Correspondence Matching for Generalizable Neural Radiance Fieldscited by 46
  30. 2021Extreme Rotation Estimation using Dense Correlation Volumescited by 42
  31. 2018Real-time Deep Pose Estimation with Geodesic Loss for Image-to-Template Rigid Registrationcited by 40
  32. 2022The 8-Point Algorithm as an Inductive Bias for Relative Pose Prediction by ViTscited by 38
  33. 2023FlowCam: Training Generalizable 3D Radiance Fields without Camera Poses via Pixel-Aligned Scene Flowcited by 38
  34. 2022Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondencecited by 32
  35. 2018RPNet: an End-to-End Network for Relative Camera Pose Estimationcited by 31
  36. 2021Deep Matching Prior: Test-Time Optimization for Dense Correspondencecited by 30
  37. 2022Generalized Differentiable RANSACcited by 30
  38. 2023Two-View Geometry Scoring Without Correspondencescited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 22; 2026: 9.

37 papers.

Sort:

By area:

  1. 2024No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Imagescited by 241
  2. 2024MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Secondscited by 136
  3. 2024FreeSplatter: Pose-free Gaussian Splatting for Sparse-view 3D Reconstructioncited by 40
  4. 2025Visual Representation Alignment for Multimodal Large Language Modelscited by 40
  5. 2025No Pose at All: Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Viewscited by 38
  6. 2024Cross-View Completion Models are Zero-shot Correspondence Estimatorscited by 30
  7. 2025Emergent Temporal Correspondences from Video Diffusion Transformerscited by 30
  8. 2024PreF3R: Pose-Free Feed-Forward 3D Gaussian Splatting from Variable-length Image Sequencecited by 26
  9. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 23
  10. 2025Emergent Outlier View Rejection in Visual Geometry Grounded Transformerscited by 19
  11. 2025C3G: Learning Compact 3D Representations with 2K Gaussianscited by 18
  12. 2025SpatialCrafter: Unleashing the Imagination of Video Diffusion Models for Scene Reconstruction from Limited Observationscited by 16
25 more
  1. 2025VicaSplat: A Single Run is All You Need for 3D Gaussian Splatting and Camera Estimation from Unposed Video Framescited by 15
  2. 20253D Scene Prompting for Scene-Consistent Camera-Controllable Video Generationcited by 13
  3. 2026TokenSplat: Token-aligned 3D Gaussian Splatting for Feed-forward Pose-free Reconstructioncited by 13
  4. 2025RegGS: Unposed Sparse Views Gaussian Splatting with 3DGS Registrationcited by 12
  5. 2025D2USt3R: Enhancing 3D Reconstruction for Dynamic Scenescited by 10
  6. 2025Vid-CamEdit: Video Camera Trajectory Editing with Generative Rendering from Estimated Geometrycited by 7
  7. 2025EcoSplat: Efficiency-controllable Feed-forward 3D Gaussian Splatting from Multi-view Imagescited by 7
  8. 2025S^4M: Boosting Semi-Supervised Instance Segmentation with SAMcited by 5
  9. 2025Text-Aware Image Restoration with Diffusion Modelscited by 5
  10. 2025SPFSplatV2: Efficient Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Viewscited by 5
  11. 2025SaLon3R: Structure-aware Long-term Generalizable 3D Reconstruction from Unposed Imagescited by 4
  12. 2026LagerNVS: Latent Geometry for Fully Neural Real-time Novel View Synthesiscited by 4
  13. 2026TORA: Topological Representation Alignment for 3D Shape Assemblycited by 4
  14. 2025Recollection from Pensieve: Novel View Synthesis via Learning from Uncalibrated Videoscited by 3
  15. 2025Stylos: Multi-View 3D Stylization with Single-Forward Gaussian Splattingcited by 3
  16. 2025G3Splat: Geometrically Consistent Generalizable Gaussian Splattingcited by 3
  17. 2026From None to All: Self-Supervised 3D Reconstruction via Novel View Synthesiscited by 2
  18. 2026No Pose, No Problem in 4D: Feed-Forward Dynamic Gaussians from Unposed Multi-View Videoscited by 2
  19. 2026Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstructioncited by 2
  20. 2026StructSplat: Generalizable 3D Gaussian Splatting from Uncalibrated Sparse Viewscited by 2
  21. 2024DynSUP: Dynamic Gaussian Splatting from An Unposed Image Paircited by 1
  22. 2026ViewSplat: View-Adaptive 3D Gaussian Splatting for Feed-Forward Synthesiscited by 1
  23. 2025Exploring Conditions for Diffusion models in Robotic Controlcited by 0
  24. 2025HybridSplat: Fast Reflection-baked Gaussian Tracing using Hybrid Splattingcited by 0
  25. 2026DVSM: Decoder-only View Synthesis Model Done Rightcited by 0