mlsanityData to 14 Oct 2026

ICON: Incremental CONfidence for Joint Pose and Radiance Field Optimization

arXiv 2401.08937 ↗First posted 17 Jan 2024cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Neural Radiance Fields (NeRF) exhibit remarkable performance for Novel View Synthesis (NVS) given a set of 2D images. However, NeRF training requires accurate camera pose for each input view, typically obtained by Structure-from-Motion (SfM) pipelines. Recent works have attempted to relax this constraint, but they still often rely on decent initial poses which they can refine. Here we aim at removing the requirement for pose initialization. We present Incremental CONfidence (ICON), an optimization procedure for training NeRFs from 2D video frames. ICON only assumes smooth camera motion to estimate initial guess for poses. Further, ICON introduces ``confidence": an adaptive measure of model quality used to dynamically reweight gradients. ICON relies on high-confidence poses to learn NeRF, and high-confidence 3D structure (as encoded by NeRF) to learn poses. We show that ICON, without prior pose initialization, achieves superior performance in both CO3D and HO3D versus methods which use SfM pose.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 60 of its 85 references that are in the map. First posted on arXiv 17 Jan 2024.

Builds on

60 of 85 references.

References · 60

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  5. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  6. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  7. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  8. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  9. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
  10. 2021Volume Rendering of Neural Implicit Surfacescited by 996
  11. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
  12. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 806
38 more
  1. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 775
  2. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
  3. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  4. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
  5. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 646
  6. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 642
  7. 2018From Coarse to Fine: Robust Hierarchical Localization at Large Scalecited by 559
  8. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 495
  9. 2016Direct Sparse Odometrycited by 485
  10. 2020INeRF: Inverting Neural Radiance Fields for Pose Estimationcited by 382
  11. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 366
  12. 2021Neural RGB-D Surface Reconstructioncited by 313
  13. 2020CosyPose: Consistent multi-view multi-object 6D pose estimationcited by 287
  14. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
  15. 2022NoPe-NeRF: Optimising Neural Radiance Field with No Pose Priorcited by 235
  16. 2021Self-Calibrating Neural Radiance Fieldscited by 204
  17. 2022MegaPose: 6D Pose Estimation of Novel Objects via Render & Comparecited by 201
  18. 2022SPARF: Neural Radiance Fields from Sparse and Noisy Posescited by 195
  19. 2022NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fieldscited by 188
  20. 2019Implicit 3D Orientation Learning for 6D Object Detection from RGB Imagescited by 170
  21. 2021GNeRF: GAN-based Neural Radiance Field without Posed Cameracited by 153
  22. 2023BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objectscited by 148
  23. 2023PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustmentcited by 147
  24. 2021NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wildcited by 136
  25. 2022Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Imagescited by 121
  26. 2020TartanVO: A Generalizable Learning-based VOcited by 108
  27. 2023Progressively Optimized Local Radiance Fields for Robust View Synthesiscited by 103
  28. 2022RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wildcited by 102
  29. 2023General In-Hand Object Rotation with Vision and Touchcited by 99
  30. 2018Fusion++: Volumetric Object-Level SLAMcited by 98
  31. 2018MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objectscited by 95
  32. 2022ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wildcited by 92
  33. 2019LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimationcited by 89
  34. 2022SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collectionscited by 89
  35. 2021BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Modelscited by 87
  36. 2022GARF: Gaussian Activated Radiance Fields for High Fidelity Reconstruction and Pose Estimationcited by 86
  37. 2022You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstrationcited by 81
  38. 2022Local-to-Global Registration for Bundle-Adjusting Neural Radiance Fieldscited by 58

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Detailed Geometry and Appearance from Opportunistic Motioncited by 0