# FusionRF: High-Fidelity Satellite Neural Radiance Fields from Multispectral and Panchromatic Acquisitions

arXiv: https://arxiv.org/abs/2409.15132 · HTML: https://mlsanity.com/paper/2409.15132/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 79 references that are in the map.
- First posted on arXiv 23 Sep 2024.

## Builds on (most cited first, 23 of 23)

- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images](https://mlsanity.com/paper/2111.13679/) (2021)
- [Deblur-NeRF: Neural Radiance Fields from Blurry Images](https://mlsanity.com/paper/2111.14292/) (2021)
- [BAD-NeRF: Bundle Adjusted Deblur Neural Radiance Fields](https://mlsanity.com/paper/2211.12853/) (2022)
- [Using satellite imagery to understand and promote sustainable development](https://mlsanity.com/paper/2010.06988/) (2020)
- [PLGSLAM: Progressive Neural Scene Represenation with Local to Global Bundle Adjustment](https://mlsanity.com/paper/2312.09866/) (2023)
- [Sat-NeRF: Learning Multi-View Satellite Photogrammetry With Transient Objects and Shadow Modeling Using RPC Cameras](https://mlsanity.com/paper/2203.08896/) (2022)
- [A Multi-Scale and Multi-Depth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening](https://mlsanity.com/paper/1712.09809/) (2017)
- [Semantic Stereo for Incidental Satellite Images](https://mlsanity.com/paper/1811.08739/) (2018)
- [Shadow Neural Radiance Fields for Multi-view Satellite Photogrammetry](https://mlsanity.com/paper/2104.09877/) (2021)
- [Pansharpening via Detail Injection Based Convolutional Neural Networks](https://mlsanity.com/paper/1806.08898/) (2018)
- [Boosting the accuracy of multi-spectral image pan-sharpening by learning a deep residual network](https://mlsanity.com/paper/1705.07556/) (2017)
- [SparseSat-NeRF: Dense Depth Supervised Neural Radiance Fields for Sparse Satellite Images](https://mlsanity.com/paper/2309.00277/) (2023)
- [TrackNeRF: Bundle Adjusting NeRF from Sparse and Noisy Views via Feature Tracks](https://mlsanity.com/paper/2408.10739/) (2024)
- [Multiscale Representation for Real-Time Anti-Aliasing Neural Rendering](https://mlsanity.com/paper/2304.10075/) (2023)
- [PGMAN: An Unsupervised Generative Multi-adversarial Network for Pan-sharpening](https://mlsanity.com/paper/2012.09054/) (2020)
- [Full-resolution quality assessment for pansharpening](https://mlsanity.com/paper/2108.06144/) (2021)
- [Unsupervised Pansharpening Based on Self-Attention Mechanism](https://mlsanity.com/paper/2006.09303/) (2020)
- [Variational Zero-shot Multispectral Pansharpening](https://mlsanity.com/paper/2407.06633/) (2024)

## Built on by (most cited first, 5 of 5)

- [Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery](https://mlsanity.com/paper/2510.15869/) (2025)
- [EOGS++: Earth Observation Gaussian Splatting with Internal Camera Refinement and Direct Panchromatic Rendering](https://mlsanity.com/paper/2511.16542/) (2025)
- [Tile and Slide : A New Framework for Scaling NeRF from Local to Global 3D Earth Observation](https://mlsanity.com/paper/2507.01631/) (2025)
- [Methods for evaluating the resolution of 3D data derived from satellite images](https://mlsanity.com/paper/2506.11876/) (2025)
- [Wrivinder: Towards Spatial Intelligence for Geo-locating Ground Images onto Satellite Imagery](https://mlsanity.com/paper/2602.14929/) (2026)

## Abstract (arXiv)

We introduce FusionRF, a novel framework for digital surface reconstruction from satellite multispectral and panchromatic images. Current work has demonstrated the increased accuracy of neural photogrammetry for surface reconstruction from optical satellite images compared to algorithmic methods. Common satellites produce both a panchromatic and multispectral image, which contain high spatial and spectral information respectively. Current neural reconstruction methods require multispectral images to be upsampled with a pansharpening method using the spatial data in the panchromatic image. However, these methods may introduce biases and hallucinations due to domain gaps. FusionRF introduces joint image fusion during optimization through a novel cross-resolution kernel that learns to resolve spatial resolution loss present in multispectral images. As input, FusionRF accepts the original multispectral and panchromatic data, eliminating the need for image preprocessing. FusionRF also leverages multimodal appearance embeddings that encode the image characteristics of each modality and view within a uniform representation. By optimizing on both modalities, FusionRF learns to fuse image modalities while performing reconstruction tasks and eliminates the need for a pansharpening preprocessing step. We evaluate our method on multispectral and panchromatic satellite images from the WorldView-3 satellite in various locations, and show that FusionRF provides an average of 17% reduction in depth reconstruction error, and renders sharp training and novel views.