# Vision-based Neural Scene Representations for Spacecraft

arXiv: https://arxiv.org/abs/2105.06405 · HTML: https://mlsanity.com/paper/2105.06405/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 54 references that are in the map.
- First posted on arXiv 11 May 2021.

## Builds on (most cited first, 30 of 30)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1812.11788/) (2018)
- [INeRF: Inverting Neural Radiance Fields for Pose Estimation](https://mlsanity.com/paper/2012.05877/) (2020)
- [SynSin: End-to-end View Synthesis from a Single Image](https://mlsanity.com/paper/1912.08804/) (2019)
- [State of the Art on Neural Rendering](https://mlsanity.com/paper/2004.03805/) (2020)
- [Learning to Estimate 3D Human Pose and Shape from a Single Color Image](https://mlsanity.com/paper/1805.04092/) (2018)
- [Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistency](https://mlsanity.com/paper/1704.06254/) (2017)
- [Escaping Plato's Cave: 3D Shape From Adversarial Rendering](https://mlsanity.com/paper/1811.11606/) (2018)
- [Semantic Image Inpainting with Deep Generative Models](https://mlsanity.com/paper/1607.07539/) (2016)
- [Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction](https://mlsanity.com/paper/1706.07036/) (2017)
- [Generative Face Completion](https://mlsanity.com/paper/1704.05838/) (2017)
- [Satellite Pose Estimation Challenge: Dataset, Competition Design and Results](https://mlsanity.com/paper/1911.02050/) (2019)
- [Real-Time, Flight-Ready, Non-Cooperative Spacecraft Pose Estimation Using Monocular Imagery](https://mlsanity.com/paper/2101.09553/) (2021)
- [AUTO3D: Novel view synthesis through unsupervisely learned variational viewpoint and global 3D representation](https://mlsanity.com/paper/2007.06620/) (2020)
- [L6DNet: Light 6 DoF Network for Robust and Precise Object Pose Estimation with Small Datasets](https://mlsanity.com/paper/2002.00911/) (2020)

## Built on by (most cited first, 12 of 12)

- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [3D Reconstruction of Non-cooperative Resident Space Objects using Instant NGP-accelerated NeRF and D-NeRF](https://mlsanity.com/paper/2301.09060/) (2023)
- [Characterizing Satellite Geometry via Accelerated 3D Gaussian Splatting](https://mlsanity.com/paper/2401.02588/) (2024)
- [Leveraging Neural Radiance Fields for Pose Estimation of an Unknown Space Object during Proximity Operations](https://mlsanity.com/paper/2405.12728/) (2024)
- [Tile and Slide : A New Framework for Scaling NeRF from Local to Global 3D Earth Observation](https://mlsanity.com/paper/2507.01631/) (2025)
- [Improved 3D Gaussian Splatting of Unknown Spacecraft Structure Using Space Environment Illumination Knowledge](https://mlsanity.com/paper/2512.23998/) (2025)
- [Computer vision tasks for intelligent aerospace missions: An overview](https://mlsanity.com/paper/2407.06513/) (2024)
- [Joint attitude estimation and 3D neural reconstruction of non-cooperative space objects](https://mlsanity.com/paper/2506.20638/) (2025)
- [Dynamic Scene 3D Reconstruction of an Uncooperative Resident Space Object](https://mlsanity.com/paper/2509.07932/) (2025)
- [On the Geometric Accuracy of Implicit and Primitive-based Representations Derived from View Rendering Constraints](https://mlsanity.com/paper/2509.10241/) (2025)
- [From Images2Mesh: A 3D Surface Reconstruction Pipeline for Non-Cooperative Space Objects](https://mlsanity.com/paper/2605.00147/) (2026)
- [NeRF-based Spacecraft Reconstruction from Monocular Imagery Under Illumination Variability and Pose Uncertainty](https://mlsanity.com/paper/2605.18447/) (2026)

## Abstract (arXiv)

In advanced mission concepts with high levels of autonomy, spacecraft need to internally model the pose and shape of nearby orbiting objects. Recent works in neural scene representations show promising results for inferring generic three-dimensional scenes from optical images. Neural Radiance Fields (NeRF) have shown success in rendering highly specular surfaces using a large number of images and their pose. More recently, Generative Radiance Fields (GRAF) achieved full volumetric reconstruction of a scene from unposed images only, thanks to the use of an adversarial framework to train a NeRF. In this paper, we compare and evaluate the potential of NeRF and GRAF to render novel views and extract the 3D shape of two different spacecraft, the Soil Moisture and Ocean Salinity satellite of ESA's Living Planet Programme and a generic cube sat. Considering the best performances of both models, we observe that NeRF has the ability to render more accurate images regarding the material specularity of the spacecraft and its pose. For its part, GRAF generates precise novel views with accurate details even when parts of the satellites are shadowed while having the significant advantage of not needing any information about the relative pose.