# MiNL: Micro-images based Neural Representation for Light Fields

arXiv: https://arxiv.org/abs/2209.08277 · HTML: https://mlsanity.com/paper/2209.08277/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 65 references that are in the map.
- First posted on arXiv 17 Sep 2022.

## Builds on (most cited first, 30 of 30)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines](https://mlsanity.com/paper/1905.00889/) (2019)
- [Learning Structured Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1608.03665/) (2016)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [NeRV: Neural Representations for Videos](https://mlsanity.com/paper/2110.13903/) (2021)
- [Net2Net: Accelerating Learning via Knowledge Transfer](https://mlsanity.com/paper/1511.05641/) (2015)
- [COIN: COmpression with Implicit Neural representations](https://mlsanity.com/paper/2103.03123/) (2021)
- [Pushing the Boundaries of View Extrapolation with Multiplane Images](https://mlsanity.com/paper/1905.00413/) (2019)
- [CodeNeRF: Disentangled Neural Radiance Fields for Object Categories](https://mlsanity.com/paper/2109.01750/) (2021)
- [View Synthesis by Appearance Flow](https://mlsanity.com/paper/1605.03557/) (2016)
- [MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesis](https://mlsanity.com/paper/2103.14910/) (2021)
- [Layer-adaptive sparsity for the Magnitude-based Pruning](https://mlsanity.com/paper/2010.07611/) (2020)
- [Dynamic Sparse Training: Find Efficient Sparse Network From Scratch With Trainable Masked Layers](https://mlsanity.com/paper/2005.06870/) (2020)
- [Learning to Synthesize a 4D RGBD Light Field from a Single Image](https://mlsanity.com/paper/1708.03292/) (2017)
- [Learning Neural Light Fields with Ray-Space Embedding Networks](https://mlsanity.com/paper/2112.01523/) (2021)
- [X-Fields: Implicit Neural View-, Light- and Time-Image Interpolation](https://mlsanity.com/paper/2010.00450/) (2020)
- [Efficient and Accurate Approximations of Nonlinear Convolutional Networks](https://mlsanity.com/paper/1411.4229/) (2014)
- [Deep Surface Light Fields](https://mlsanity.com/paper/1810.06514/) (2018)
- [Fast and Accurate Depth Estimation from Sparse Light Fields](https://mlsanity.com/paper/1812.06856/) (2018)
- [SADN: Learned Light Field Image Compression with Spatial-Angular Decorrelation](https://mlsanity.com/paper/2202.10837/) (2022)
- [High performance ultra-low-precision convolutions on mobile devices](https://mlsanity.com/paper/1712.02427/) (2017)
- [Light Field Implicit Representation for Flexible Resolution Reconstruction](https://mlsanity.com/paper/2112.00185/) (2021)

## Built on by (most cited first, 1 of 1)

- [Scale Guided Hypernetwork for Blind Super-Resolution Image Quality Assessment](https://mlsanity.com/paper/2306.02398/) (2023)

## Abstract (arXiv)

Traditional representations for light fields can be separated into two types: explicit representation and implicit representation. Unlike explicit representation that represents light fields as Sub-Aperture Images (SAIs) based arrays or Micro-Images (MIs) based lenslet images, implicit representation treats light fields as neural networks, which is inherently a continuous representation in contrast to discrete explicit representation. However, at present almost all the implicit representations for light fields utilize SAIs to train an MLP to learn a pixel-wise mapping from 4D spatial-angular coordinate to pixel colors, which is neither compact nor of low complexity. Instead, in this paper we propose MiNL, a novel MI-wise implicit neural representation for light fields that train an MLP + CNN to learn a mapping from 2D MI coordinates to MI colors. Given the micro-image's coordinate, MiNL outputs the corresponding micro-image's RGB values. Light field encoding in MiNL is just training a neural network to regress the micro-images and the decoding process is a simple feedforward operation. Compared with common pixel-wise implicit representation, MiNL is more compact and efficient that has faster decoding speed (×80~180 speed-up) as well as better visual quality (1~4dB PSNR improvement on average).