# NeRF-VIO: Map-Based Visual-Inertial Odometry with Initialization Leveraging Neural Radiance Fields

arXiv: https://arxiv.org/abs/2503.07952 · HTML: https://mlsanity.com/paper/2503.07952/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 39 references that are in the map.
- First posted on arXiv 11 Mar 2025.

## Builds on (most cited first, 18 of 18)

- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [INeRF: Inverting Neural Radiance Fields for Pose Estimation](https://mlsanity.com/paper/2012.05877/) (2020)
- [NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAM](https://mlsanity.com/paper/2302.03594/) (2023)
- [Deep Learning Features at Scale for Visual Place Recognition](https://mlsanity.com/paper/1701.05105/) (2017)
- [Loc-NeRF: Monte Carlo Localization using Neural Radiance Fields](https://mlsanity.com/paper/2209.09050/) (2022)
- [maplab: An Open Framework for Research in Visual-inertial Mapping and Localization](https://mlsanity.com/paper/1711.10250/) (2017)
- [maplab 2.0 -- A Modular and Multi-Modal Mapping Framework](https://mlsanity.com/paper/2212.00654/) (2022)
- [Efficient 2D-3D Matching for Multi-Camera Visual Localization](https://mlsanity.com/paper/1809.06445/) (2018)
- [NeRF-VINS: A Real-time Neural Radiance Field Map-based Visual-Inertial Navigation System](https://mlsanity.com/paper/2309.09295/) (2023)
- [Keyframe-Based Visual-Inertial Online SLAM with Relocalization](https://mlsanity.com/paper/1702.02175/) (2017)
- [PL-CVIO: Point-Line Cooperative Visual-Inertial Odometry](https://mlsanity.com/paper/2311.05717/) (2023)
- [Visual Navigation Using Sparse Optical Flow and Time-to-Transit](https://mlsanity.com/paper/2111.09669/) (2021)
- [Map-based Visual-Inertial Localization: Consistency and Complexity](https://mlsanity.com/paper/2204.12173/) (2022)

## Built on by (most cited first, 3 of 3)

- [VIGS-SLAM: Visual Inertial Gaussian Splatting SLAM](https://mlsanity.com/paper/2512.02293/) (2025)
- [VBGS-SLAM: Variational Bayesian Gaussian Splatting Simultaneous Localization and Mapping](https://mlsanity.com/paper/2604.02696/) (2026)
- [An Efficient Approach for Muscle Segmentation and 3D Reconstruction Using Keypoint Tracking in MRI Scan](https://mlsanity.com/paper/2507.08690/) (2025)

## Abstract (arXiv)

A prior map serves as a foundational reference for localization in context-aware applications such as augmented reality (AR). Providing valuable contextual information about the environment, the prior map is a vital tool for mitigating drift. In this paper, we propose a map-based visual-inertial localization algorithm (NeRF-VIO) with initialization using neural radiance fields (NeRF). Our algorithm utilizes a multilayer perceptron model and redefines the loss function as the geodesic distance on (SE(3)), ensuring the invariance of the initialization model under a frame change within (mathfrakse(3)). The evaluation demonstrates that our model outperforms existing NeRF-based initialization solution in both accuracy and efficiency. By integrating a two-stage update mechanism within a multi-state constraint Kalman filter (MSCKF) framework, the state of NeRF-VIO is constrained by both captured images from an onboard camera and rendered images from a pre-trained NeRF model. The proposed algorithm is validated using a real-world AR dataset, the results indicate that our two-stage update pipeline outperforms MSCKF across all data sequences.