# nerf2nerf: Pairwise Registration of Neural Radiance Fields

arXiv: https://arxiv.org/abs/2211.01600 · HTML: https://mlsanity.com/paper/2211.01600/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 67 references that are in the map.
- First posted on arXiv 3 Nov 2022.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [INeRF: Inverting Neural Radiance Fields for Pose Estimation](https://mlsanity.com/paper/2012.05877/) (2020)
- [Decomposing NeRF for Editing via Feature Field Distillation](https://mlsanity.com/paper/2205.15585/) (2022)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [Deep Closest Point: Learning Representations for Point Cloud Registration](https://mlsanity.com/paper/1905.03304/) (2019)
- [PREDATOR: Registration of 3D Point Clouds with Low Overlap](https://mlsanity.com/paper/2011.13005/) (2020)
- [TEASER: Fast and Certifiable Point Cloud Registration](https://mlsanity.com/paper/2001.07715/) (2020)
- [PointNetLK: Robust & Efficient Point Cloud Registration using PointNet](https://mlsanity.com/paper/1903.05711/) (2019)
- [Self-Calibrating Neural Radiance Fields](https://mlsanity.com/paper/2108.13826/) (2021)
- [COTR: Correspondence Transformer for Matching Across Images](https://mlsanity.com/paper/2103.14167/) (2021)
- [The Perfect Match: 3D Point Cloud Matching with Smoothed Densities](https://mlsanity.com/paper/1811.06879/) (2018)
- [NeRF--: Neural Radiance Fields Without Known Camera Parameters](https://mlsanity.com/paper/2102.07064/) (2021)
- [NASA: Neural Articulated Shape Approximation](https://mlsanity.com/paper/1912.03207/) (2019)
- [PRNet: Self-Supervised Learning for Partial-to-Partial Registration](https://mlsanity.com/paper/1910.12240/) (2019)
- [3D Neural Scene Representations for Visuomotor Control](https://mlsanity.com/paper/2107.04004/) (2021)
- [ReLU Fields: The Little Non-linearity That Could](https://mlsanity.com/paper/2205.10824/) (2022)
- [NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields](https://mlsanity.com/paper/2203.01913/) (2022)
- [Guaranteed Outlier Removal for Point Cloud Registration with Correspondences](https://mlsanity.com/paper/1711.10209/) (2017)
- [Volume Rendering Digest (for NeRF)](https://mlsanity.com/paper/2209.02417/) (2022)
- [Tactile Object Pose Estimation from the First Touch with Geometric Contact Rendering](https://mlsanity.com/paper/2012.05205/) (2020)
- [PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization](https://mlsanity.com/paper/1505.07427/) (2015)

## Built on by (most cited first, 30 of 39)

- [RadSplat: Radiance Field-Informed Gaussian Splatting for Robust Real-Time Rendering with 900+ FPS](https://mlsanity.com/paper/2403.13806/) (2024)
- [LoopSplat: Loop Closure by Registering 3D Gaussian Splats](https://mlsanity.com/paper/2408.10154/) (2024)
- [SparsePose: Sparse-View Camera Pose Regression and Refinement](https://mlsanity.com/paper/2211.16991/) (2022)
- [DOGS: Distributed-Oriented Gaussian Splatting for Large-Scale 3D Reconstruction Via Gaussian Consensus](https://mlsanity.com/paper/2405.13943/) (2024)
- [Learning a More Continuous Zero Level Set in Unsigned Distance Fields through Level Set Projection](https://mlsanity.com/paper/2308.11441/) (2023)
- [DReg-NeRF: Deep Registration for Neural Radiance Fields](https://mlsanity.com/paper/2308.09386/) (2023)
- [GaussReg: Fast 3D Registration with Gaussian Splatting](https://mlsanity.com/paper/2407.05254/) (2024)
- [SOAC: Spatio-Temporal Overlap-Aware Multi-Sensor Calibration using Neural Radiance Fields](https://mlsanity.com/paper/2311.15803/) (2023)
- [NEAT: Distilling 3D Wireframes from Neural Attraction Fields](https://mlsanity.com/paper/2307.10206/) (2023)
- [GridPull: Towards Scalability in Learning Implicit Representations from 3D Point Clouds](https://mlsanity.com/paper/2308.13175/) (2023)
- [Neural radiance fields in the industrial and robotics domain: applications, research opportunities and use cases](https://mlsanity.com/paper/2308.07118/) (2023)
- [BANF: Band-limited Neural Fields for Levels of Detail Reconstruction](https://mlsanity.com/paper/2404.13024/) (2024)
- [ZeroGS: Training 3D Gaussian Splatting from Unposed Images](https://mlsanity.com/paper/2411.15779/) (2024)
- [Volumetric Rendering with Baked Quadrature Fields](https://mlsanity.com/paper/2312.02202/) (2023)
- [Evaluating Alternatives to SFM Point Cloud Initialization for Gaussian Splatting](https://mlsanity.com/paper/2404.12547/) (2024)
- [Canonical Factors for Hybrid Neural Fields](https://mlsanity.com/paper/2308.15461/) (2023)
- [SIREN: Semantic, Initialization-Free Registration of Multi-Robot Gaussian Splatting Maps](https://mlsanity.com/paper/2502.06519/) (2025)
- [Clean-NeRF: Reformulating NeRF to account for View-Dependent Observations](https://mlsanity.com/paper/2303.14707/) (2023)
- [C-NERF: Representing Scene Changes as Directional Consistency Difference-based NeRF](https://mlsanity.com/paper/2312.02751/) (2023)
- [Blending Distributed NeRFs with Tri-stage Robust Pose Optimization](https://mlsanity.com/paper/2405.02880/) (2024)
- [NeRFuser: Large-Scale Scene Representation by NeRF Fusion](https://mlsanity.com/paper/2305.13307/) (2023)
- [PhotoReg: Photometrically Registering 3D Gaussian Splatting Models](https://mlsanity.com/paper/2410.05044/) (2024)
- [Scalable and High-Quality Neural Implicit Representation for 3D Reconstruction](https://mlsanity.com/paper/2501.08577/) (2025)
- [TEGLO: High Fidelity Canonical Texture Mapping from Single-View Images](https://mlsanity.com/paper/2303.13743/) (2023)
- [SCALAR-NeRF: SCAlable LARge-scale Neural Radiance Fields for Scene Reconstruction](https://mlsanity.com/paper/2311.16657/) (2023)
- [Reg-NF: Efficient Registration of Implicit Surfaces within Neural Fields](https://mlsanity.com/paper/2402.09722/) (2024)
- [Learning 3D Scene Analogies with Neural Contextual Scene Maps](https://mlsanity.com/paper/2503.15897/) (2025)
- [Holistic Large-Scale Scene Reconstruction via Mixed Gaussian Splatting](https://mlsanity.com/paper/2505.23280/) (2025)
- [Automated 3D-GS Registration and Fusion via Skeleton Alignment and Gaussian-Adaptive Features](https://mlsanity.com/paper/2507.20480/) (2025)
- [Registering Neural Radiance Fields as 3D Density Images](https://mlsanity.com/paper/2305.12843/) (2023)

## Abstract (arXiv)

We introduce a technique for pairwise registration of neural fields that extends classical optimization-based local registration (i.e. ICP) to operate on Neural Radiance Fields (NeRF) -- neural 3D scene representations trained from collections of calibrated images. NeRF does not decompose illumination and color, so to make registration invariant to illumination, we introduce the concept of a ''surface field'' -- a field distilled from a pre-trained NeRF model that measures the likelihood of a point being on the surface of an object. We then cast nerf2nerf registration as a robust optimization that iteratively seeks a rigid transformation that aligns the surface fields of the two scenes. We evaluate the effectiveness of our technique by introducing a dataset of pre-trained NeRF scenes -- our synthetic scenes enable quantitative evaluations and comparisons to classical registration techniques, while our real scenes demonstrate the validity of our technique in real-world scenarios. Additional results available at: https://nerf2nerf.github.io