# Neural Image Representations for Multi-Image Fusion and Layer Separation

arXiv: https://arxiv.org/abs/2108.01199 · HTML: https://mlsanity.com/paper/2108.01199/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 2 Aug 2021.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [Learned Initializations for Optimizing Coordinate-Based Neural Representations](https://mlsanity.com/paper/2012.02189/) (2020)
- [Adversarial Generation of Continuous Images](https://mlsanity.com/paper/2011.12026/) (2020)
- [Attention-guided Network for Ghost-free High Dynamic Range Imaging](https://mlsanity.com/paper/1904.10293/) (2019)
- ["Double-DIP": Unsupervised Image Decomposition via Coupled Deep-Image-Priors](https://mlsanity.com/paper/1812.00467/) (2018)
- [Image Generators with Conditionally-Independent Pixel Synthesis](https://mlsanity.com/paper/2011.13775/) (2020)
- [Single Image Reflection Separation with Perceptual Losses](https://mlsanity.com/paper/1806.05376/) (2018)
- [Handheld Multi-Frame Super-Resolution](https://mlsanity.com/paper/1905.03277/) (2019)
- [Deep Burst Super-Resolution](https://mlsanity.com/paper/2101.10997/) (2021)
- [Robust Video Content Alignment and Compensation for Rain Removal in a CNN Framework](https://mlsanity.com/paper/1803.10433/) (2018)
- [Omnimatte: Associating Objects and Their Effects in Video](https://mlsanity.com/paper/2105.06993/) (2021)
- [Spatially-Adaptive Pixelwise Networks for Fast Image Translation](https://mlsanity.com/paper/2012.02992/) (2020)
- [Learning to See Through Obstructions](https://mlsanity.com/paper/2004.01180/) (2020)
- [FastDeRain: A Novel Video Rain Streak Removal Method Using Directional Gradient Priors](https://mlsanity.com/paper/1803.07487/) (2018)
- [The Visual Centrifuge: Model-Free Layered Video Representations](https://mlsanity.com/paper/1812.01461/) (2018)
- [NTIRE 2021 Challenge on Burst Super-Resolution: Methods and Results](https://mlsanity.com/paper/2106.03839/) (2021)
- [Learning to See Through Obstructions with Layered Decomposition](https://mlsanity.com/paper/2008.04902/) (2020)
- [Large-Scale Query-by-Image Video Retrieval Using Bloom Filters](https://mlsanity.com/paper/1604.07939/) (2016)

## Built on by (most cited first, 18 of 18)

- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Local-to-Global Registration for Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2211.11505/) (2022)
- [Bidirectional Multi-Scale Implicit Neural Representations for Image Deraining](https://mlsanity.com/paper/2404.01547/) (2024)
- [Neural Spline Fields for Burst Image Fusion and Layer Separation](https://mlsanity.com/paper/2312.14235/) (2023)
- [DDMI: Domain-Agnostic Latent Diffusion Models for Synthesizing High-Quality Implicit Neural Representations](https://mlsanity.com/paper/2401.12517/) (2024)
- [DC-Gaussian: Improving 3D Gaussian Splatting for Reflective Dash Cam Videos](https://mlsanity.com/paper/2405.17705/) (2024)
- [DeNVeR: Deformable Neural Vessel Representations for Unsupervised Video Vessel Segmentation](https://mlsanity.com/paper/2406.01591/) (2024)
- [Indoor Obstacle Discovery on Reflective Ground via Monocular Camera](https://mlsanity.com/paper/2401.01445/) (2024)
- [NaRCan: Natural Refined Canonical Image with Integration of Diffusion Prior for Video Editing](https://mlsanity.com/paper/2406.06523/) (2024)
- [Neural Light Spheres for Implicit Image Stitching and View Synthesis](https://mlsanity.com/paper/2409.17924/) (2024)
- [Blind Video Super-Resolution based on Implicit Kernels](https://mlsanity.com/paper/2503.07856/) (2025)
- [Continuous Representation Methods, Theories, and Applications: An Overview and Perspectives](https://mlsanity.com/paper/2505.15222/) (2025)
- [Implicit neural representations for joint decomposition and registration of gene expression images in the marmoset brain](https://mlsanity.com/paper/2308.04039/) (2023)
- [Generating visual explanations from deep networks using implicit neural representations](https://mlsanity.com/paper/2501.11784/) (2025)
- [HyperNVD: Accelerating Neural Video Decomposition via Hypernetworks](https://mlsanity.com/paper/2503.17276/) (2025)
- [SuperF: Neural Implicit Fields for Multi-Image Super-Resolution](https://mlsanity.com/paper/2512.09115/) (2025)
- [Learning Implicit Neural Degradation Representation for Unpaired Image Dehazing](https://mlsanity.com/paper/2511.13110/) (2025)
- [AMIF: Authorizable Medical Image Fusion Model with Built-in Authentication](https://mlsanity.com/paper/2603.24296/) (2026)

## Abstract (arXiv)

We propose a framework for aligning and fusing multiple images into a single view using neural image representations (NIRs), also known as implicit or coordinate-based neural representations. Our framework targets burst images that exhibit camera ego motion and potential changes in the scene. We describe different strategies for alignment depending on the nature of the scene motion -- namely, perspective planar (i.e., homography), optical flow with minimal scene change, and optical flow with notable occlusion and disocclusion. With the neural image representation, our framework effectively combines multiple inputs into a single canonical view without the need for selecting one of the images as a reference frame. We demonstrate how to use this multi-frame fusion framework for various layer separation tasks. The code and results are available at https://shnnam.github.io/research/nir.