# Physics-Informed Ensemble Representation for Light-Field Image Super-Resolution

arXiv: https://arxiv.org/abs/2305.20006 · HTML: https://mlsanity.com/paper/2305.20006/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 58 references that are in the map.
- First posted on arXiv 31 May 2023.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Learning-Based View Synthesis for Light Field Cameras](https://mlsanity.com/paper/1609.02974/) (2016)
- [Disentangling Light Fields for Super-Resolution and Disparity Estimation](https://mlsanity.com/paper/2202.10603/) (2022)
- [Spatial-Angular Interaction for Light Field Image Super-Resolution](https://mlsanity.com/paper/1912.07849/) (2019)
- [Light Field Spatial Super-resolution via Deep Combinatorial Geometry Embedding and Structural Consistency Regularization](https://mlsanity.com/paper/2004.02215/) (2020)
- [Light Field Image Super-Resolution with Transformers](https://mlsanity.com/paper/2108.07597/) (2021)
- [Light Field Image Super-Resolution Using Deformable Convolution](https://mlsanity.com/paper/2007.03535/) (2020)
- [Detail-Preserving Transformer for Light Field Image Super-Resolution](https://mlsanity.com/paper/2201.00346/) (2022)
- [Learning Non-Local Spatial-Angular Correlation for Light Field Image Super-Resolution](https://mlsanity.com/paper/2302.08058/) (2023)
- [Light Field Reconstruction Using Shearlet Transform](https://mlsanity.com/paper/1509.08969/) (2015)
- [Learning Light Field Angular Super-Resolution via a Geometry-Aware Network](https://mlsanity.com/paper/2002.11263/) (2020)
- [Light Field Super-Resolution Via Graph-Based Regularization](https://mlsanity.com/paper/1701.02141/) (2017)
- [Revisiting Light Field Rendering with Deep Anti-Aliasing Neural Network](https://mlsanity.com/paper/2104.06797/) (2021)
- [Spatial-Angular Attention Network for Light Field Reconstruction](https://mlsanity.com/paper/2007.02252/) (2020)
- [Real-World Light Field Image Super-Resolution via Degradation Modulation](https://mlsanity.com/paper/2206.06214/) (2022)
- [Dense Dual-Attention Network for Light Field Image Super-Resolution](https://mlsanity.com/paper/2110.12114/) (2021)

## Abstract (arXiv)

Recent learning-based approaches have achieved significant progress in light field (LF) image super-resolution (SR) by exploring convolution-based or transformer-based network structures. However, LF imaging has many intrinsic physical priors that have not been fully exploited. In this paper, we analyze the coordinate transformation of the LF imaging process to reveal the geometric relationship in the LF images. Based on such geometric priors, we introduce a new LF subspace of virtual-slit images (VSI) that provide sub-pixel information complementary to sub-aperture images. To leverage the abundant correlation across the four-dimensional data with manageable complexity, we propose learning ensemble representation of all C42 LF subspaces for more effective feature extraction. To super-resolve image structures from undersampled LF data, we propose a geometry-aware decoder, named EPIXformer, which constrains the transformer's operational searching regions with a LF physical prior. Experimental results on both spatial and angular SR tasks demonstrate that the proposed method outperforms other state-of-the-art schemes, especially in handling various disparities.