# I See-Through You: A Framework for Removing Foreground Occlusion in Both Sparse and Dense Light Field Images

arXiv: https://arxiv.org/abs/2301.06392 · HTML: https://mlsanity.com/paper/2301.06392/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 43 references that are in the map.
- First posted on arXiv 16 Jan 2023.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [Recurrent Feature Reasoning for Image Inpainting](https://mlsanity.com/paper/2008.03737/) (2020)
- [GAN Memory with No Forgetting](https://mlsanity.com/paper/2006.07543/) (2020)
- [Image Inpainting with Learnable Bidirectional Attention Maps](https://mlsanity.com/paper/1909.00968/) (2019)
- [Learning Parallax Attention for Stereo Image Super-Resolution](https://mlsanity.com/paper/1903.05784/) (2019)
- [Light Field Image Super-Resolution Using Deformable Convolution](https://mlsanity.com/paper/2007.03535/) (2020)
- [Piggyback GAN: Efficient Lifelong Learning for Image Conditioned Generation](https://mlsanity.com/paper/2104.11939/) (2021)
- [DeOccNet: Learning to See Through Foreground Occlusions in Light Fields](https://mlsanity.com/paper/1912.04459/) (2019)
- [DUT-LFSaliency: Versatile Dataset and Light Field-to-RGB Saliency Detection](https://mlsanity.com/paper/2012.15124/) (2020)

## Built on by (most cited first, 2 of 2)

- [Unveiling the Potential of Spike Streams for Foreground Occlusion Removal from Densely Continuous Views](https://mlsanity.com/paper/2307.00821/) (2023)
- [Vision-Reasoning-Guided Occlusion Removal from Light Fields](https://mlsanity.com/paper/2606.19985/) (2026)

## Abstract (arXiv)

Light field (LF) camera captures rich information from a scene. Using the information, the LF de-occlusion (LF-DeOcc) task aims to reconstruct the occlusion-free center view image. Existing LF-DeOcc studies mainly focus on the sparsely sampled (sparse) LF images where most of the occluded regions are visible in other views due to the large disparity. In this paper, we expand LF-DeOcc in more challenging datasets, densely sampled (dense) LF images, which are taken by a micro-lens-based portable LF camera. Due to the small disparity ranges of dense LF images, most of the background regions are invisible in any view. To apply LF-DeOcc in both LF datasets, we propose a framework, ISTY, which is defined and divided into three roles: (1) extract LF features, (2) define the occlusion, and (3) inpaint occluded regions. By dividing the framework into three specialized components according to the roles, the development and analysis can be easier. Furthermore, an explainable intermediate representation, an occlusion mask, can be obtained in the proposed framework. The occlusion mask is useful for comprehensive analysis of the model and other applications by manipulating the mask. In experiments, qualitative and quantitative results show that the proposed framework outperforms state-of-the-art LF-DeOcc methods in both sparse and dense LF datasets.