# Learning Non-Local Spatial-Angular Correlation for Light Field Image Super-Resolution

arXiv: https://arxiv.org/abs/2302.08058 · HTML: https://mlsanity.com/paper/2302.08058/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 87 references that are in the map.
- First posted on arXiv 16 Feb 2023.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [NeX: Real-time View Synthesis with Neural Basis Expansion](https://mlsanity.com/paper/2103.05606/) (2021)
- [Attention to Scale: Scale-aware Semantic Image Segmentation](https://mlsanity.com/paper/1511.03339/) (2015)
- [Learning-Based View Synthesis for Light Field Cameras](https://mlsanity.com/paper/1609.02974/) (2016)
- [Interpreting Super-Resolution Networks with Local Attribution Maps](https://mlsanity.com/paper/2011.11036/) (2020)
- [Light Field Neural Rendering](https://mlsanity.com/paper/2112.09687/) (2021)
- [Learning Signed Distance Field for Multi-view Surface Reconstruction](https://mlsanity.com/paper/2108.09964/) (2021)
- [Rethinking Alignment in Video Super-Resolution Transformers](https://mlsanity.com/paper/2207.08494/) (2022)
- [Learning Parallax Attention for Stereo Image Super-Resolution](https://mlsanity.com/paper/1903.05784/) (2019)
- [Learning Neural Light Fields with Ray-Space Embedding Networks](https://mlsanity.com/paper/2112.01523/) (2021)
- [Disentangling Light Fields for Super-Resolution and Disparity Estimation](https://mlsanity.com/paper/2202.10603/) (2022)
- [Parallax Attention for Unsupervised Stereo Correspondence Learning](https://mlsanity.com/paper/2009.08250/) (2020)
- [Spatial-Angular Interaction for Light Field Image Super-Resolution](https://mlsanity.com/paper/1912.07849/) (2019)
- [Light Field Spatial Super-resolution via Deep Combinatorial Geometry Embedding and Structural Consistency Regularization](https://mlsanity.com/paper/2004.02215/) (2020)
- [Light Field Image Super-Resolution with Transformers](https://mlsanity.com/paper/2108.07597/) (2021)

## Built on by (most cited first, 21 of 21)

- [Incorporating Degradation Estimation in Light Field Spatial Super-Resolution](https://mlsanity.com/paper/2405.07012/) (2024)
- [NTIRE 2023 Challenge on Light Field Image Super-Resolution: Dataset, Methods and Results](https://mlsanity.com/paper/2304.10415/) (2023)
- [LFMamba: Light Field Image Super-Resolution with State Space Model](https://mlsanity.com/paper/2406.12463/) (2024)
- [Probabilistic-based Feature Embedding of 4-D Light Fields for Compressive Imaging and Denoising](https://mlsanity.com/paper/2306.08836/) (2023)
- [LFSRDiff: Light Field Image Super-Resolution via Diffusion Models](https://mlsanity.com/paper/2311.16517/) (2023)
- [Toward Real-World Light Field Super-Resolution](https://mlsanity.com/paper/2305.18994/) (2023)
- [Beyond Subspace Isolation: Many-to-Many Transformer for Light Field Image Super-resolution](https://mlsanity.com/paper/2401.00740/) (2024)
- [LGFN: Lightweight Light Field Image Super-Resolution using Local Convolution Modulation and Global Attention Feature Extraction](https://mlsanity.com/paper/2409.17759/) (2024)
- [L2FMamba: Lightweight Light Field Image Super-Resolution with State Space Model](https://mlsanity.com/paper/2503.19253/) (2025)
- [OAFuser: Towards Omni-Aperture Fusion for Light Field Semantic Segmentation](https://mlsanity.com/paper/2307.15588/) (2023)
- [Enhancing Underwater Imaging with 4-D Light Fields: Dataset and Method](https://mlsanity.com/paper/2408.17339/) (2024)
- [MaskBlur: Spatial and Angular Data Augmentation for Light Field Image Super-Resolution](https://mlsanity.com/paper/2410.06478/) (2024)
- [Deep Learning Empowered Super-Resolution: A Comprehensive Survey and Future Prospects](https://mlsanity.com/paper/2509.22692/) (2025)
- [Physics-Informed Ensemble Representation for Light-Field Image Super-Resolution](https://mlsanity.com/paper/2305.20006/) (2023)
- [Self-supervised Learning-based Reconstruction of High-resolution 4D Light Fields](https://mlsanity.com/paper/2402.19020/) (2024)
- [Less is More: Skim Transformer for Light Field Image Super-resolution](https://mlsanity.com/paper/2407.15329/) (2024)
- [Exploring Non-Local Spatial-Angular Correlations with a Hybrid Mamba-Transformer Framework for Light Field Super-Resolution](https://mlsanity.com/paper/2509.04824/) (2025)
- [Enhancing Underwater Light Field Images via Global Geometry-aware Diffusion Process](https://mlsanity.com/paper/2601.21179/) (2026)
- [RASLF: Representation-Aware State Space Model for Light Field Super-Resolution](https://mlsanity.com/paper/2603.16243/) (2026)
- [GTF: Omnidirectional EPI Transformer for Light Field Super-Resolution](https://mlsanity.com/paper/2605.04581/) (2026)
- [Slope-Guided Mamba and Angular-Refined Transformer for Light Field Super-Resolution](https://mlsanity.com/paper/2607.00965/) (2026)

## Abstract (arXiv)

Exploiting spatial-angular correlation is crucial to light field (LF) image super-resolution (SR), but is highly challenging due to its non-local property caused by the disparities among LF images. Although many deep neural networks (DNNs) have been developed for LF image SR and achieved continuously improved performance, existing methods cannot well leverage the long-range spatial-angular correlation and thus suffer a significant performance drop when handling scenes with large disparity variations. In this paper, we propose a simple yet effective method to learn the non-local spatial-angular correlation for LF image SR. In our method, we adopt the epipolar plane image (EPI) representation to project the 4D spatial-angular correlation onto multiple 2D EPI planes, and then develop a Transformer network with repetitive self-attention operations to learn the spatial-angular correlation by modeling the dependencies between each pair of EPI pixels. Our method can fully incorporate the information from all angular views while achieving a global receptive field along the epipolar line. We conduct extensive experiments with insightful visualizations to validate the effectiveness of our method. Comparative results on five public datasets show that our method not only achieves state-of-the-art SR performance, but also performs robust to disparity variations. Code is publicly available at https://github.com/ZhengyuLiang24/EPIT.