# Geo-NI: Geometry-aware Neural Interpolation for Light Field Rendering

arXiv: https://arxiv.org/abs/2206.09736 · HTML: https://mlsanity.com/paper/2206.09736/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 77 references that are in the map.
- First posted on arXiv 20 Jun 2022.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [Making Convolutional Networks Shift-Invariant Again](https://mlsanity.com/paper/1904.11486/) (2019)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [Single-View View Synthesis with Multiplane Images](https://mlsanity.com/paper/2004.11364/) (2020)
- [DeepView: View Synthesis with Learned Gradient Descent](https://mlsanity.com/paper/1906.07316/) (2019)
- [NeX: Real-time View Synthesis with Neural Basis Expansion](https://mlsanity.com/paper/2103.05606/) (2021)
- [Pushing the Boundaries of View Extrapolation with Multiplane Images](https://mlsanity.com/paper/1905.00413/) (2019)
- [DeepStereo: Learning to Predict New Views from the World's Imagery](https://mlsanity.com/paper/1506.06825/) (2015)
- [Object Detectors Emerge in Deep Scene CNNs](https://mlsanity.com/paper/1412.6856/) (2014)
- [HDR image reconstruction from a single exposure using deep CNNs](https://mlsanity.com/paper/1710.07480/) (2017)
- [Learning-Based View Synthesis for Light Field Cameras](https://mlsanity.com/paper/1609.02974/) (2016)
- [Light Field Neural Rendering](https://mlsanity.com/paper/2112.09687/) (2021)
- [Single-View View Synthesis in the Wild with Learned Adaptive Multiplane Images](https://mlsanity.com/paper/2205.11733/) (2022)
- [Disentangling Light Fields for Super-Resolution and Disparity Estimation](https://mlsanity.com/paper/2202.10603/) (2022)
- [Do Convnets Learn Correspondence?](https://mlsanity.com/paper/1411.1091/) (2014)
- [A System for Acquiring, Processing, and Rendering Panoramic Light Field Stills for Virtual Reality](https://mlsanity.com/paper/1810.08860/) (2018)
- [High-dimensional Dense Residual Convolutional Neural Network for Light Field Reconstruction](https://mlsanity.com/paper/1910.01426/) (2019)
- [Light Field Reconstruction Using Shearlet Transform](https://mlsanity.com/paper/1509.08969/) (2015)
- [Light Field Reconstruction Using Convolutional Network on EPI and Extended Applications](https://mlsanity.com/paper/2103.13043/) (2021)
- [Learning Light Field Angular Super-Resolution via a Geometry-Aware Network](https://mlsanity.com/paper/2002.11263/) (2020)

## Built on by (most cited first, 1 of 1)

- [CostFilter-AD: Enhancing Anomaly Detection through Matching Cost Filtering](https://mlsanity.com/paper/2505.01476/) (2025)

## Abstract (arXiv)

In this paper, we present a Geometry-aware Neural Interpolation (Geo-NI) framework for light field rendering. Previous learning-based approaches either rely on the capability of neural networks to perform direct interpolation, which we dubbed Neural Interpolation (NI), or explore scene geometry for novel view synthesis, also known as Depth Image-Based Rendering (DIBR). Instead, we incorporate the ideas behind these two kinds of approaches by launching the NI with a novel DIBR pipeline. Specifically, the proposed Geo-NI first performs NI using input light field sheared by a set of depth hypotheses. Then the DIBR is implemented by assigning the sheared light fields with a novel reconstruction cost volume according to the reconstruction quality under different depth hypotheses. The reconstruction cost is interpreted as a blending weight to render the final output light field by blending the reconstructed light fields along the dimension of depth hypothesis. By combining the superiorities of NI and DIBR, the proposed Geo-NI is able to render views with large disparity with the help of scene geometry while also reconstruct non-Lambertian effect when depth is prone to be ambiguous. Extensive experiments on various datasets demonstrate the superior performance of the proposed geometry-aware light field rendering framework.