# Texture-enhanced Light Field Super-resolution with Spatio-Angular Decomposition Kernels

arXiv: https://arxiv.org/abs/2111.04069 · HTML: https://mlsanity.com/paper/2111.04069/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 74 references that are in the map.
- First posted on arXiv 7 Nov 2021.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis](https://mlsanity.com/paper/1612.07919/) (2016)
- [Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation](https://mlsanity.com/paper/1611.05250/) (2016)
- [Feedback Network for Image Super-Resolution](https://mlsanity.com/paper/1903.09814/) (2019)
- [Learning-Based View Synthesis for Light Field Cameras](https://mlsanity.com/paper/1609.02974/) (2016)
- [EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth from Light Field Images](https://mlsanity.com/paper/1804.02379/) (2018)
- [Spatial-Angular Interaction for Light Field Image Super-Resolution](https://mlsanity.com/paper/1912.07849/) (2019)
- [Light Field Spatial Super-resolution via Deep Combinatorial Geometry Embedding and Structural Consistency Regularization](https://mlsanity.com/paper/2004.02215/) (2020)
- [A 4D Light-Field Dataset and CNN Architectures for Material Recognition](https://mlsanity.com/paper/1608.06985/) (2016)
- [Accurate Light Field Depth Estimation with Superpixel Regularization over Partially Occluded Regions](https://mlsanity.com/paper/1708.01964/) (2017)
- [Light Field Reconstruction Using Convolutional Network on EPI and Extended Applications](https://mlsanity.com/paper/2103.13043/) (2021)
- [Analyzing Perception-Distortion Tradeoff using Enhanced Perceptual Super-resolution Network](https://mlsanity.com/paper/1811.00344/) (2018)
- [Light Field Super-Resolution Via Graph-Based Regularization](https://mlsanity.com/paper/1701.02141/) (2017)
- [High-Order Residual Network for Light Field Super-Resolution](https://mlsanity.com/paper/2003.13094/) (2020)
- [Light Field Compression with Disparity Guided Sparse Coding based on Structural Key Views](https://mlsanity.com/paper/1610.03684/) (2016)
- [The Level Weighted Structural Similarity Loss: A Step Away from the MSE](https://mlsanity.com/paper/1904.13362/) (2019)
- [Spatial and Angular Resolution Enhancement of Light Fields Using Convolutional Neural Networks](https://mlsanity.com/paper/1707.00815/) (2017)
- [Efficient Light Field Reconstruction via Spatio-Angular Dense Network](https://mlsanity.com/paper/2108.03635/) (2021)

## Built on by (most cited first, 3 of 3)

- [Beyond Subspace Isolation: Many-to-Many Transformer for Light Field Image Super-resolution](https://mlsanity.com/paper/2401.00740/) (2024)
- [Light field Rectification based on relative pose estimation](https://mlsanity.com/paper/2201.12533/) (2022)
- [Less is More: Skim Transformer for Light Field Image Super-resolution](https://mlsanity.com/paper/2407.15329/) (2024)

## Abstract (arXiv)

Despite the recent progress in light field super-resolution (LFSR) achieved by convolutional neural networks, the correlation information of light field (LF) images has not been sufficiently studied and exploited due to the complexity of 4D LF data. To cope with such high-dimensional LF data, most of the existing LFSR methods resorted to decomposing it into lower dimensions and subsequently performing optimization on the decomposed sub-spaces. However, these methods are inherently limited as they neglected the characteristics of the decomposition operations and only utilized a limited set of LF sub-spaces ending up failing to sufficiently extract spatio-angular features and leading to a performance bottleneck. To overcome these limitations, in this paper, we comprehensively discover the potentials of LF decomposition and propose a novel concept of decomposition kernels. In particular, we systematically unify the decomposition operations of various sub-spaces into a series of such decomposition kernels, which are incorporated into our proposed Decomposition Kernel Network (DKNet) for comprehensive spatio-angular feature extraction. The proposed DKNet is experimentally verified to achieve considerable improvements compared with the state-of-the-art methods. To further improve DKNet in producing more visually pleasing LFSR results, based on the VGG network, we propose a LFVGG loss to guide the Texture-Enhanced DKNet (TE-DKNet) to generate rich authentic textures and enhance LF images' visual quality significantly. We also propose an indirect evaluation metric by taking advantage of LF material recognition to objectively assess the perceptual enhancement brought by the LFVGG loss.