# A 4D Light-Field Dataset and CNN Architectures for Material Recognition

arXiv: https://arxiv.org/abs/1608.06985 · HTML: https://mlsanity.com/paper/1608.06985/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Aug 2016.

## Built on by (most cited first, 26 of 26)

- [A Dense Material Segmentation Dataset for Indoor and Outdoor Scene Parsing](https://mlsanity.com/paper/2207.10614/) (2022)
- [Light Field Reconstruction Using Convolutional Network on EPI and Extended Applications](https://mlsanity.com/paper/2103.13043/) (2021)
- [SAMa: Material-aware 3D Selection and Segmentation](https://mlsanity.com/paper/2411.19322/) (2024)
- [Time-Efficient Light-Field Acquisition Using Coded Aperture and Events](https://mlsanity.com/paper/2403.07244/) (2024)
- [Dense Dual-Attention Network for Light Field Image Super-Resolution](https://mlsanity.com/paper/2110.12114/) (2021)
- [Probabilistic-based Feature Embedding of 4-D Light Fields for Compressive Imaging and Denoising](https://mlsanity.com/paper/2306.08836/) (2023)
- [The joint role of geometry and illumination on material recognition](https://mlsanity.com/paper/2101.02496/) (2021)
- [Light Field Reconstruction via Deep Adaptive Fusion of Hybrid Lenses](https://mlsanity.com/paper/2102.07085/) (2021)
- [Acquiring a Dynamic Light Field through a Single-Shot Coded Image](https://mlsanity.com/paper/2204.12089/) (2022)
- [Unsupervised Learning of Depth Estimation and Visual Odometry for Sparse Light Field Cameras](https://mlsanity.com/paper/2103.11322/) (2021)
- [Texture-enhanced Light Field Super-resolution with Spatio-Angular Decomposition Kernels](https://mlsanity.com/paper/2111.04069/) (2021)
- [Beyond Subspace Isolation: Many-to-Many Transformer for Light Field Image Super-resolution](https://mlsanity.com/paper/2401.00740/) (2024)
- [Fine-Grained Spatially Varying Material Selection in Images](https://mlsanity.com/paper/2506.09023/) (2025)
- [Efficient Light Field Reconstruction via Spatio-Angular Dense Network](https://mlsanity.com/paper/2108.03635/) (2021)
- [Spectral Reconstruction and Disparity from Spatio-Spectrally Coded Light Fields via Multi-Task Deep Learning](https://mlsanity.com/paper/2103.10179/) (2021)
- [Dynamic Fusion Network For Light Field Depth Estimation](https://mlsanity.com/paper/2104.05969/) (2021)
- [An Extensible Multimodal Multi-task Object Dataset with Materials](https://mlsanity.com/paper/2305.14352/) (2023)
- [LiFMCR: Dataset and Benchmark for Light Field Multi-Camera Registration](https://mlsanity.com/paper/2510.13729/) (2025)
- [Coded-E2LF: Coded Aperture Light Field Imaging from Events](https://mlsanity.com/paper/2602.22620/) (2026)
- [A Learned Compact and Editable Light Field Representation](https://mlsanity.com/paper/2103.11314/) (2021)
- [Learning-based Spatial and Angular Information Separation for Light Field Compression](https://mlsanity.com/paper/2304.06322/) (2023)
- [Reconstructing Continuous Light Field From Single Coded Image](https://mlsanity.com/paper/2311.09646/) (2023)
- [Less is More: Skim Transformer for Light Field Image Super-resolution](https://mlsanity.com/paper/2407.15329/) (2024)
- [Light-Field Dataset for Disparity Based Depth Estimation](https://mlsanity.com/paper/2511.05866/) (2025)
- [FMMC: Harnessing the Power of Foundation Models for Accurate Material Classification](https://mlsanity.com/paper/2603.17390/) (2026)
- [VSANet: View-aware Sparse Attention Network for Light Field Image Denoising](https://mlsanity.com/paper/2606.24737/) (2026)

## Abstract (arXiv)

We introduce a new light-field dataset of materials, and take advantage of the recent success of deep learning to perform material recognition on the 4D light-field. Our dataset contains 12 material categories, each with 100 images taken with a Lytro Illum, from which we extract about 30,000 patches in total. To the best of our knowledge, this is the first mid-size dataset for light-field images. Our main goal is to investigate whether the additional information in a light-field (such as multiple sub-aperture views and view-dependent reflectance effects) can aid material recognition. Since recognition networks have not been trained on 4D images before, we propose and compare several novel CNN architectures to train on light-field images. In our experiments, the best performing CNN architecture achieves a 7% boost compared with 2D image classification (70% to 77%). These results constitute important baselines that can spur further research in the use of CNNs for light-field applications. Upon publication, our dataset also enables other novel applications of light-fields, including object detection, image segmentation and view interpolation.