# IFFNeRF: Initialisation Free and Fast 6DoF pose estimation from a single image and a NeRF model

arXiv: https://arxiv.org/abs/2403.12682 · HTML: https://mlsanity.com/paper/2403.12682/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 27 references that are in the map.
- First posted on arXiv 19 Mar 2024.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [INeRF: Inverting Neural Radiance Fields for Pose Estimation](https://mlsanity.com/paper/2012.05877/) (2020)
- [kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation](https://mlsanity.com/paper/1903.06684/) (2019)
- [UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policy](https://mlsanity.com/paper/2303.00938/) (2023)
- [Loc-NeRF: Monte Carlo Localization using Neural Radiance Fields](https://mlsanity.com/paper/2209.09050/) (2022)
- [Parallel Inversion of Neural Radiance Fields for Robust Pose Estimation](https://mlsanity.com/paper/2210.10108/) (2022)
- [CROSSFIRE: Camera Relocalization On Self-Supervised Features from an Implicit Representation](https://mlsanity.com/paper/2303.04869/) (2023)
- [LabelFusion: A Pipeline for Generating Ground Truth Labels for Real RGBD Data of Cluttered Scenes](https://mlsanity.com/paper/1707.04796/) (2017)
- [VM-NeRF: Tackling Sparsity in NeRF with View Morphing](https://mlsanity.com/paper/2210.04214/) (2022)
- [RGBD2lux: Dense light intensity estimation with an RGBD sensor](https://mlsanity.com/paper/1809.07558/) (2018)

## Built on by (most cited first, 5 of 5)

- [6DGS: 6D Pose Estimation from a Single Image and a 3D Gaussian Splatting Model](https://mlsanity.com/paper/2407.15484/) (2024)
- [Neural Fields in Robotics: A Survey](https://mlsanity.com/paper/2410.20220/) (2024)
- [SplatPose: Geometry-Aware 6-DoF Pose Estimation from Single RGB Image via 3D Gaussian Splatting](https://mlsanity.com/paper/2503.05174/) (2025)
- [Metropolis-Hastings Sampling for 3D Gaussian Reconstruction](https://mlsanity.com/paper/2506.12945/) (2025)
- [HistReNeRF: Historic Image Relocalisation within Contemporary Neural Radiance Field Reconstructions](https://mlsanity.com/paper/2608.15420/) (2026)

## Abstract (arXiv)

We introduce IFFNeRF to estimate the six degrees-of-freedom (6DoF) camera pose of a given image, building on the Neural Radiance Fields (NeRF) formulation. IFFNeRF is specifically designed to operate in real-time and eliminates the need for an initial pose guess that is proximate to the sought solution. IFFNeRF utilizes the Metropolis-Hasting algorithm to sample surface points from within the NeRF model. From these sampled points, we cast rays and deduce the color for each ray through pixel-level view synthesis. The camera pose can then be estimated as the solution to a Least Squares problem by selecting correspondences between the query image and the resulting bundle. We facilitate this process through a learned attention mechanism, bridging the query image embedding with the embedding of parameterized rays, thereby matching rays pertinent to the image. Through synthetic and real evaluation settings, we show that our method can improve the angular and translation error accuracy by 80.1% and 67.3%, respectively, compared to iNeRF while performing at 34fps on consumer hardware and not requiring the initial pose guess.