# Leveraging Neural Radiance Fields for Pose Estimation of an Unknown Space Object during Proximity Operations

arXiv: https://arxiv.org/abs/2405.12728 · HTML: https://mlsanity.com/paper/2405.12728/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 21 May 2024.

## Builds on (most cited first, 21 of 21)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [K-Planes: Explicit Radiance Fields in Space, Time, and Appearance](https://mlsanity.com/paper/2301.10241/) (2023)
- [INeRF: Inverting Neural Radiance Fields for Pose Estimation](https://mlsanity.com/paper/2012.05877/) (2020)
- [NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields](https://mlsanity.com/paper/2210.13641/) (2022)
- [Vision-Only Robot Navigation in a Neural Radiance World](https://mlsanity.com/paper/2110.00168/) (2021)
- [FoV-NeRF: Foveated Neural Radiance Fields for Virtual Reality](https://mlsanity.com/paper/2103.16365/) (2021)
- [Satellite Pose Estimation Challenge: Dataset, Competition Design and Results](https://mlsanity.com/paper/1911.02050/) (2019)
- [SPEED+: Next-Generation Dataset for Spacecraft Pose Estimation across Domain Gap](https://mlsanity.com/paper/2110.03101/) (2021)
- [Satellite Pose Estimation with Deep Landmark Regression and Nonlinear Pose Refinement](https://mlsanity.com/paper/1908.11542/) (2019)
- [Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering](https://mlsanity.com/paper/1907.04298/) (2019)
- [EfficientPose: An efficient, accurate and scalable end-to-end 6D multi object pose estimation approach](https://mlsanity.com/paper/2011.04307/) (2020)
- [Robust Multi-Task Learning and Online Refinement for Spacecraft Pose Estimation across Domain Gap](https://mlsanity.com/paper/2203.04275/) (2022)
- [Pose Estimation for Non-Cooperative Spacecraft Rendezvous Using Convolutional Neural Networks](https://mlsanity.com/paper/1809.07238/) (2018)
- [A Survey on Deep Learning-Based Monocular Spacecraft Pose Estimation: Current State, Limitations and Prospects](https://mlsanity.com/paper/2305.07348/) (2023)
- [Towards Robust Learning-Based Pose Estimation of Noncooperative Spacecraft](https://mlsanity.com/paper/1909.00392/) (2019)
- [Robotic Testbed for Rendezvous and Optical Navigation: Multi-Source Calibration and Machine Learning Use Cases](https://mlsanity.com/paper/2108.05529/) (2021)
- [Vision-based Neural Scene Representations for Spacecraft](https://mlsanity.com/paper/2105.06405/) (2021)
- [3D Reconstruction of Non-cooperative Resident Space Objects using Instant NGP-accelerated NeRF and D-NeRF](https://mlsanity.com/paper/2301.09060/) (2023)
- [Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review](https://mlsanity.com/paper/2402.11141/) (2024)

## Built on by (most cited first, 7 of 7)

- [Domain Generalization for 6D Pose Estimation Through NeRF-based Image Synthesis](https://mlsanity.com/paper/2407.10762/) (2024)
- [Domain Generalization for In-Orbit 6D Pose Estimation](https://mlsanity.com/paper/2406.11743/) (2024)
- [NeRF-based Visualization of 3D Cues Supporting Data-Driven Spacecraft Pose Estimation](https://mlsanity.com/paper/2509.14890/) (2025)
- [Joint attitude estimation and 3D neural reconstruction of non-cooperative space objects](https://mlsanity.com/paper/2506.20638/) (2025)
- [On the Geometric Accuracy of Implicit and Primitive-based Representations Derived from View Rendering Constraints](https://mlsanity.com/paper/2509.10241/) (2025)
- [NeRF-based Spacecraft Reconstruction from Monocular Imagery Under Illumination Variability and Pose Uncertainty](https://mlsanity.com/paper/2605.18447/) (2026)
- [CAD-Free Learning of Spacecraft Pose Estimators via NeRF-Based Augmentations](https://mlsanity.com/paper/2605.19649/) (2026)

## Abstract (arXiv)

We address the estimation of the 6D pose of an unknown target spacecraft relative to a monocular camera, a key step towards the autonomous rendezvous and proximity operations required by future Active Debris Removal missions. We present a novel method that enables an "off-the-shelf" spacecraft pose estimator, which is supposed to known the target CAD model, to be applied on an unknown target. Our method relies on an in-the wild NeRF, i.e., a Neural Radiance Field that employs learnable appearance embeddings to represent varying illumination conditions found in natural scenes. We train the NeRF model using a sparse collection of images that depict the target, and in turn generate a large dataset that is diverse both in terms of viewpoint and illumination. This dataset is then used to train the pose estimation network. We validate our method on the Hardware-In-the-Loop images of SPEED+ that emulate lighting conditions close to those encountered on orbit. We demonstrate that our method successfully enables the training of an off-the-shelf spacecraft pose estimation network from a sparse set of images. Furthermore, we show that a network trained using our method performs similarly to a model trained on synthetic images generated using the CAD model of the target.