# NeRF-VINS: A Real-time Neural Radiance Field Map-based Visual-Inertial Navigation System

arXiv: https://arxiv.org/abs/2309.09295 · HTML: https://mlsanity.com/paper/2309.09295/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2023.

## Built on by (most cited first, 7 of 7)

- [CG-SLAM: Efficient Dense RGB-D SLAM in a Consistent Uncertainty-aware 3D Gaussian Field](https://mlsanity.com/paper/2403.16095/) (2024)
- [CityGS-X: A Scalable Architecture for Efficient and Geometrically Accurate Large-Scale Scene Reconstruction](https://mlsanity.com/paper/2503.23044/) (2025)
- [Thermal-NeRF: Neural Radiance Fields from an Infrared Camera](https://mlsanity.com/paper/2403.10340/) (2024)
- [Neural Radiance Field in Autonomous Driving: A Survey](https://mlsanity.com/paper/2404.13816/) (2024)
- [VIGS-SLAM: Visual Inertial Gaussian Splatting SLAM](https://mlsanity.com/paper/2512.02293/) (2025)
- [NeRF-VIO: Map-Based Visual-Inertial Odometry with Initialization Leveraging Neural Radiance Fields](https://mlsanity.com/paper/2503.07952/) (2025)
- [VBGS-SLAM: Variational Bayesian Gaussian Splatting Simultaneous Localization and Mapping](https://mlsanity.com/paper/2604.02696/) (2026)

## Abstract (arXiv)

Achieving efficient and consistent localization a prior map remains challenging in robotics. Conventional keyframe-based approaches often suffers from sub-optimal viewpoints due to limited field of view (FOV) and/or constrained motion, thus degrading the localization performance. To address this issue, we design a real-time tightly-coupled Neural Radiance Fields (NeRF)-aided visual-inertial navigation system (VINS). In particular, by effectively leveraging the NeRF's potential to synthesize novel views, the proposed NeRF-VINS overcomes the limitations of traditional keyframe-based maps (with limited views) and optimally fuses IMU, monocular images, and synthetically rendered images within an efficient filter-based framework. This tightly-coupled fusion enables efficient 3D motion tracking with bounded errors. We extensively compare the proposed NeRF-VINS against the state-of-the-art methods that use prior map information and demonstrate its ability to perform real-time localization, at over 10 Hz, on a resource-constrained Jetson AGX Orin embedded platform.