# Local-to-Global Registration for Bundle-Adjusting Neural Radiance Fields

arXiv: https://arxiv.org/abs/2211.11505 · HTML: https://mlsanity.com/paper/2211.11505/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 57 references that are in the map.
- First posted on arXiv 21 Nov 2022.

## Builds on (most cited first, 30 of 35)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines](https://mlsanity.com/paper/1905.00889/) (2019)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video](https://mlsanity.com/paper/2201.04127/) (2022)
- [INeRF: Inverting Neural Radiance Fields for Pose Estimation](https://mlsanity.com/paper/2012.05877/) (2020)
- [Kornia: an Open Source Differentiable Computer Vision Library for PyTorch](https://mlsanity.com/paper/1910.02190/) (2019)
- [NeX: Real-time View Synthesis with Neural Basis Expansion](https://mlsanity.com/paper/2103.05606/) (2021)
- [Self-Calibrating Neural Radiance Fields](https://mlsanity.com/paper/2108.13826/) (2021)
- [NeRF--: Neural Radiance Fields Without Known Camera Parameters](https://mlsanity.com/paper/2102.07064/) (2021)
- [GNeRF: GAN-based Neural Radiance Field without Posed Camera](https://mlsanity.com/paper/2103.15606/) (2021)
- [Layered Neural Atlases for Consistent Video Editing](https://mlsanity.com/paper/2109.11418/) (2021)
- [Deep Image Homography Estimation](https://mlsanity.com/paper/1606.03798/) (2016)
- [Hallucinated Neural Radiance Fields in the Wild](https://mlsanity.com/paper/2111.15246/) (2021)
- [HDR-NeRF: High Dynamic Range Neural Radiance Fields](https://mlsanity.com/paper/2111.14451/) (2021)
- [GARF: Gaussian Activated Radiance Fields for High Fidelity Reconstruction and Pose Estimation](https://mlsanity.com/paper/2204.05735/) (2022)
- [Deep Regression on Manifolds: A 3D Rotation Case Study](https://mlsanity.com/paper/2103.16317/) (2021)
- [Deep Homography Estimation for Dynamic Scenes](https://mlsanity.com/paper/2004.02132/) (2020)
- [SiNeRF: Sinusoidal Neural Radiance Fields for Joint Pose Estimation and Scene Reconstruction](https://mlsanity.com/paper/2210.04553/) (2022)
- [UV Volumes for Real-time Rendering of Editable Free-view Human Performance](https://mlsanity.com/paper/2203.14402/) (2022)

## Built on by (most cited first, 30 of 58)

- [Advances in 3D Generation: A Survey](https://mlsanity.com/paper/2401.17807/) (2024)
- [Masked Space-Time Hash Encoding for Efficient Dynamic Scene Reconstruction](https://mlsanity.com/paper/2310.17527/) (2023)
- [Instant Gaussian Stream: Fast and Generalizable Streaming of Dynamic Scene Reconstruction via Gaussian Splatting](https://mlsanity.com/paper/2503.16979/) (2025)
- [Feat2GS: Probing Visual Foundation Models with Gaussian Splatting](https://mlsanity.com/paper/2412.09606/) (2024)
- [DiSR-NeRF: Diffusion-Guided View-Consistent Super-Resolution NeRF](https://mlsanity.com/paper/2404.00874/) (2024)
- [PoRF: Pose Residual Field for Accurate Neural Surface Reconstruction](https://mlsanity.com/paper/2310.07449/) (2023)
- [NViST: In the Wild New View Synthesis from a Single Image with Transformers](https://mlsanity.com/paper/2312.08568/) (2023)
- [NIS-SLAM: Neural Implicit Semantic RGB-D SLAM for 3D Consistent Scene Understanding](https://mlsanity.com/paper/2407.20853/) (2024)
- [SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wild](https://mlsanity.com/paper/2401.10171/) (2024)
- [NeRF Director: Revisiting View Selection in Neural Volume Rendering](https://mlsanity.com/paper/2406.08839/) (2024)
- [Fast High Dynamic Range Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2401.06052/) (2024)
- [UP-NeRF: Unconstrained Pose-Prior-Free Neural Radiance Fields](https://mlsanity.com/paper/2311.03784/) (2023)
- [Implicit-Zoo: A Large-Scale Dataset of Neural Implicit Functions for 2D Images and 3D Scenes](https://mlsanity.com/paper/2406.17438/) (2024)
- [TrackNeRF: Bundle Adjusting NeRF from Sparse and Noisy Views via Feature Tracks](https://mlsanity.com/paper/2408.10739/) (2024)
- [USB-NeRF: Unrolling Shutter Bundle Adjusted Neural Radiance Fields](https://mlsanity.com/paper/2310.02687/) (2023)
- [CBARF: Cascaded Bundle-Adjusting Neural Radiance Fields from Imperfect Camera Poses](https://mlsanity.com/paper/2310.09776/) (2023)
- [3R-GS: Best Practice in Optimizing Camera Poses Along with 3DGS](https://mlsanity.com/paper/2504.04294/) (2025)
- [NeAI: A Pre-convoluted Representation for Plug-and-Play Neural Ambient Illumination](https://mlsanity.com/paper/2304.08757/) (2023)
- [ZeroGS: Training 3D Gaussian Splatting from Unposed Images](https://mlsanity.com/paper/2411.15779/) (2024)
- [SOGS: Second-Order Anchor for Advanced 3D Gaussian Splatting](https://mlsanity.com/paper/2503.07476/) (2025)
- [Generic Objects as Pose Probes for Few-shot View Synthesis](https://mlsanity.com/paper/2408.16690/) (2024)
- [BAA-NGP: Bundle-Adjusting Accelerated Neural Graphics Primitives](https://mlsanity.com/paper/2306.04166/) (2023)
- [Root Pose Decomposition Towards Generic Non-rigid 3D Reconstruction with Monocular Videos](https://mlsanity.com/paper/2308.10089/) (2023)
- [Efficient Ray Sampling for Radiance Fields Reconstruction](https://mlsanity.com/paper/2308.15547/) (2023)
- [E3NeRF: Efficient Event-Enhanced Neural Radiance Fields from Blurry Images](https://mlsanity.com/paper/2408.01840/) (2024)
- [SCARF: Scalable Continual Learning Framework for Memory-efficient Multiple Neural Radiance Fields](https://mlsanity.com/paper/2409.04482/) (2024)
- [NaRCan: Natural Refined Canonical Image with Integration of Diffusion Prior for Video Editing](https://mlsanity.com/paper/2406.06523/) (2024)
- [MC-NeRF: Multi-Camera Neural Radiance Fields for Multi-Camera Image Acquisition Systems](https://mlsanity.com/paper/2309.07846/) (2023)
- [Residual-NeRF: Learning Residual NeRFs for Transparent Object Manipulation](https://mlsanity.com/paper/2405.06181/) (2024)
- [Invertible Neural Warp for NeRF](https://mlsanity.com/paper/2407.12354/) (2024)

## Abstract (arXiv)

Neural Radiance Fields (NeRF) have achieved photorealistic novel views synthesis; however, the requirement of accurate camera poses limits its application. Despite analysis-by-synthesis extensions for jointly learning neural 3D representations and registering camera frames exist, they are susceptible to suboptimal solutions if poorly initialized. We propose L2G-NeRF, a Local-to-Global registration method for bundle-adjusting Neural Radiance Fields: first, a pixel-wise flexible alignment, followed by a frame-wise constrained parametric alignment. Pixel-wise local alignment is learned in an unsupervised way via a deep network which optimizes photometric reconstruction errors. Frame-wise global alignment is performed using differentiable parameter estimation solvers on the pixel-wise correspondences to find a global transformation. Experiments on synthetic and real-world data show that our method outperforms the current state-of-the-art in terms of high-fidelity reconstruction and resolving large camera pose misalignment. Our module is an easy-to-use plugin that can be applied to NeRF variants and other neural field applications. The Code and supplementary materials are available at https://rover-xingyu.github.io/L2G-NeRF/.