# Learning-based Spatial and Angular Information Separation for Light Field Compression

arXiv: https://arxiv.org/abs/2304.06322 · HTML: https://mlsanity.com/paper/2304.06322/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 44 references that are in the map.
- First posted on arXiv 13 Apr 2023.

## Builds on (most cited first, 13 of 13)

- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Training with Quantization Noise for Extreme Model Compression](https://mlsanity.com/paper/2004.07320/) (2020)
- [A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers](https://mlsanity.com/paper/1804.03294/) (2018)
- [Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks](https://mlsanity.com/paper/1810.03982/) (2018)
- [Learning for Video Compression with Hierarchical Quality and Recurrent Enhancement](https://mlsanity.com/paper/2003.01966/) (2020)
- [Convolutional Neural Network Pruning with Structural Redundancy Reduction](https://mlsanity.com/paper/2104.03438/) (2021)
- [Learning for Video Compression with Recurrent Auto-Encoder and Recurrent Probability Model](https://mlsanity.com/paper/2006.13560/) (2020)
- [Towards Efficient Tensor Decomposition-Based DNN Model Compression with Optimization Framework](https://mlsanity.com/paper/2107.12422/) (2021)
- [TransCut: Transparent Object Segmentation from a Light-Field Image](https://mlsanity.com/paper/1511.06853/) (2015)
- [A 4D Light-Field Dataset and CNN Architectures for Material Recognition](https://mlsanity.com/paper/1608.06985/) (2016)
- [DCFNet: Deep Neural Network with Decomposed Convolutional Filters](https://mlsanity.com/paper/1802.04145/) (2018)
- [Occlusion-Aware Cost Constructor for Light Field Depth Estimation](https://mlsanity.com/paper/2203.01576/) (2022)
- [Geometry-Aware Graph Transforms for Light Field Compact Representation](https://mlsanity.com/paper/1903.03556/) (2019)

## Abstract (arXiv)

Light fields are a type of image data that capture both spatial and angular scene information by recording light rays emitted by a scene from different orientations. In this context, spatial information is defined as features that remain static regardless of perspectives, while angular information refers to features that vary between viewpoints. We propose a novel neural network that, by design, can separate angular and spatial information of a light field. The network represents spatial information using spatial kernels shared among all Sub-Aperture Images (SAIs), and angular information using sets of angular kernels for each SAI. To further improve the representation capability of the network without increasing parameter number, we also introduce angular kernel allocation and kernel tensor decomposition mechanisms. Extensive experiments demonstrate the benefits of information separation: when applied to the compression task, our network outperforms other state-of-the-art methods by a large margin. And angular information can be easily transferred to other scenes for rendering dense views, showing the successful separation and the potential use case for the view synthesis task. We plan to release the code upon acceptance of the paper to encourage further research on this topic.