mlsanityData to 14 Oct 2026

Scene Coordinate Reconstruction: Posing of Image Collections via Incremental Learning of a Relocalizer

arXiv 2404.14351 ↗First posted 22 Apr 2024cs.CVcited by 73

Not on the map. Why?

Abstract (arXiv)

We address the task of estimating camera parameters from a set of images depicting a scene. Popular feature-based structure-from-motion (SfM) tools solve this task by incremental reconstruction: they repeat triangulation of sparse 3D points and registration of more camera views to the sparse point cloud. We re-interpret incremental structure-from-motion as an iterated application and refinement of a visual relocalizer, that is, of a method that registers new views to the current state of the reconstruction. This perspective allows us to investigate alternative visual relocalizers that are not rooted in local feature matching. We show that scene coordinate regression, a learning-based relocalization approach, allows us to build implicit, neural scene representations from unposed images. Different from other learning-based reconstruction methods, we do not require pose priors nor sequential inputs, and we optimize efficiently over thousands of images. In many cases, our method, ACE0, estimates camera poses with an accuracy close to feature-based SfM, as demonstrated by novel view synthesis. Project page: https://nianticlabs.github.io/acezero/

Key facts. Cited by 73 papers in the map (data through 14 Oct 2026). Builds on 58 of its 111 references that are in the map. First posted on arXiv 22 Apr 2024.

Builds on

58 of 111 references.

References · 58

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  3. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  5. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  6. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  7. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  8. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
  9. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  10. 2017Deep Image Priorcited by 1,098
  11. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 984
  12. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
38 more
  1. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
  2. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
  3. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 642
  4. 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 600
  5. 2023LightGlue: Local Feature Matching at Light Speedcited by 590
  6. 2018From Coarse to Fine: Robust Hierarchical Localization at Large Scalecited by 559
  7. 2020INeRF: Inverting Neural Radiance Fields for Pose Estimationcited by 382
  8. 2019D2-Net: A Trainable CNN for Joint Detection and Description of Local Featurescited by 326
  9. 2016DSAC - Differentiable RANSAC for Camera Localizationcited by 266
  10. 2022NoPe-NeRF: Optimising Neural Radiance Field with No Pose Priorcited by 235
  11. 2020Image Matching across Wide Baselines: From Paper to Practicecited by 225
  12. 2021Self-Calibrating Neural Radiance Fieldscited by 204
  13. 2016DeMoN: Depth and Motion Network for Learning Monocular Stereocited by 201
  14. 2021Back to the Feature: Learning Robust Camera Localization from Pixels to Posecited by 199
  15. 2019Understanding the Limitations of CNN-based Absolute Camera Pose Regressioncited by 177
  16. 2020Visual Camera Re-Localization from RGB and RGB-D Images Using DSACcited by 176
  17. 2021GNeRF: GAN-based Neural Radiance Field without Posed Cameracited by 153
  18. 2018CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAMcited by 152
  19. 2017Learning Less is More - 6D Camera Localization via 3D Surface Regressioncited by 146
  20. 2024Cameras as Rays: Pose Estimation via Ray Diffusioncited by 131
  21. 2022Map-free Visual Relocalization: Metric Pose Relative to a Single Imagecited by 114
  22. 2023Accelerated Coordinate Encoding: Learning to Relocalize in Minutes using RGB and Posescited by 107
  23. 2018PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Imagecited by 105
  24. 2019Hierarchical Scene Coordinate Classification and Regression for Visual Localizationcited by 93
  25. 2023RelPose++: Recovering 6D Poses from Sparse-view Observationscited by 87
  26. 2022DFNet: Enhance Absolute Pose Regression with Direct Feature Matchingcited by 82
  27. 2017Camera Relocalization by Computing Pairwise Relative Poses Using Convolutional Neural Networkcited by 79
  28. 2023Detector-Free Structure from Motioncited by 71
  29. 2019Expert Sample Consensus Applied to Camera Re-Localizationcited by 70
  30. 2019DeepSFM: Structure From Motion Via Deep Bundle Adjustmentcited by 70
  31. 2021Deep Regression on Manifolds: A 3D Rotation Case Studycited by 67
  32. 2021On the Limits of Pseudo Ground Truth in Visual Camera Re-localisationcited by 63
  33. 2019To Learn or Not to Learn: Visual Localization from Essential Matricescited by 61
  34. 2021Direct-PoseNet: Absolute Pose Regression with Photometric Consistencycited by 61
  35. 2022Parallel Inversion of Neural Radiance Fields for Robust Pose Estimationcited by 55
  36. 2022SparsePose: Sparse-View Camera Pose Regression and Refinementcited by 53
  37. 2022SiNeRF: Sinusoidal Neural Radiance Fields for Joint Pose Estimation and Scene Reconstructioncited by 49
  38. 2023CROSSFIRE: Camera Relocalization On Self-Supervised Features from an Implicit Representationcited by 45

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 13; 2025: 40; 2026: 20.

73 papers.

Sort:

By area:

  1. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  2. 20243D Reconstruction with Spatial Memorycited by 277
  3. 2025AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Viewscited by 243
  4. 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 228
  5. 2024MASt3R-SfM: a Fully-Integrated Solution for Unconstrained Structure-from-Motioncited by 162
  6. 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 141
  7. 2025TTT3R: 3D Reconstruction as Test-Time Trainingcited by 129
  8. 2025Easi3R: Estimating Disentangled Motion from DUSt3R Without Trainingcited by 91
  9. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  10. 2024FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descentcited by 61
  11. 2025Light3R-SfM: Towards Feed-forward Structure-from-Motioncited by 47
  12. 2025No Pose at All: Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Viewscited by 40
38 more
  1. 2025AerialMegaDepth: Learning Aerial-Ground Reconstruction and View Synthesiscited by 39
  2. 2025Human3R: Everyone Everywhere All at Oncecited by 37
  3. 2025On-the-fly Reconstruction for Large-Scale Novel View Synthesis from Unposed Imagescited by 36
  4. 2024Feat2GS: Probing Visual Foundation Models with Gaussian Splattingcited by 27
  5. 2025MP-SfM: Monocular Surface Priors for Robust Structure-from-Motioncited by 26
  6. 2026VGGT-Omegacited by 26
  7. 2024General Place Recognition Survey: Towards Real-World Autonomycited by 23
  8. 2024From an Image to a Scene: Learning to Imagine the World from a Million 360 Videoscited by 23
  9. 2024MAtCha Gaussians: Atlas of Charts for High-Quality Geometry and Photorealism From Sparse Viewscited by 21
  10. 2024Doppelgangers++: Improved Visual Disambiguation with Geometric 3D Featurescited by 20
  11. 2025SAIL-Recon: Large SfM by Augmenting Scene Regression with Localizationcited by 20
  12. 2025AMB3R: Accurate Feed-forward Metric-scale 3D Reconstruction with Backendcited by 20
  13. 2026VGG-T3: Offline Feed-Forward 3D Reconstruction at Scalecited by 18
  14. 2025R-SCoRe: Revisiting Scene Coordinate Regression for Robust Large-Scale Visual Localizationcited by 17
  15. 2025Dense-SfM: Structure from Motion with Dense Consistent Matchingcited by 15
  16. 2025HAMSt3R: Human-Aware Multi-view Stereo 3D Reconstructioncited by 15
  17. 2025True Self-Supervised Novel View Synthesis is Transferablecited by 15
  18. 2025DiffusionSfM: Predicting Structure and Motion via Ray Origin and Endpoint Diffusioncited by 14
  19. 2024Generative Multiview Relighting for 3D Reconstruction under Extreme Illumination Variationcited by 13
  20. 20253R-GS: Best Practice in Optimizing Camera Poses Along with 3DGScited by 11
  21. 2025Rig3R: Rig-Aware Conditioning for Learned 3D Reconstructioncited by 11
  22. 2024ZeroGS: Training 3D Gaussian Splatting from Unposed Imagescited by 10
  23. 2025Can Video Diffusion Model Reconstruct 4D Geometry?cited by 10
  24. 2025Regist3R: Incremental Registration with Stereo Foundation Modelcited by 9
  25. 2025EgoM2P: Egocentric Multimodal Multitask Pretrainingcited by 9
  26. 2025ACE-G: Improving Generalization of Scene Coordinate Regression Through Query Pre-Trainingcited by 9
  27. 2025Graph-Guided Scene Reconstruction from Images with 3D Gaussian Splattingcited by 8
  28. 2025FastMap: Revisiting Structure from Motion through First-Order Optimizationcited by 8
  29. 2025Understanding Multi-View Transformerscited by 8
  30. 2025SPFSplatV2: Efficient Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Viewscited by 7
  31. 2025Image as an IMU: Estimating Camera Motion from a Single Motion-Blurred Imagecited by 5
  32. 2026Glob3R: Global Structure-from-Motion with 3D Foundation Modelscited by 4
  33. 2025Explaining Human Preferences via Metrics for Structured 3D Reconstructioncited by 3
  34. 2025EAG3R: Event-Augmented 3D Geometry Estimation for Dynamic and Extreme-Lighting Scenescited by 3
  35. 2026Marginalized Bundle Adjustment: Multi-View Camera Pose from Monocular Depth Estimatescited by 3
  36. 2025Targetless LiDAR-Camera Calibration with Neural Gaussian Splattingcited by 2
  37. 2025Scene Coordinate Reconstruction Priorscited by 2
  38. 2026From None to All: Self-Supervised 3D Reconstruction via Novel View Synthesiscited by 2

50 most cited of 73 All 73 (JSON)