# Dynamic Fusion Network For Light Field Depth Estimation

arXiv: https://arxiv.org/abs/2104.05969 · HTML: https://mlsanity.com/paper/2104.05969/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 45 references that are in the map.
- First posted on arXiv 13 Apr 2021.

## Builds on (most cited first, 19 of 19)

- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving](https://mlsanity.com/paper/1505.00256/) (2015)
- [Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image](https://mlsanity.com/paper/1709.07492/) (2017)
- [Learning-Based View Synthesis for Light Field Cameras](https://mlsanity.com/paper/1609.02974/) (2016)
- [T2Net: Synthetic-to-Realistic Translation for Solving Single-Image Depth Estimation Tasks](https://mlsanity.com/paper/1808.01454/) (2018)
- [Learning monocular depth estimation infusing traditional stereo knowledge](https://mlsanity.com/paper/1904.04144/) (2019)
- [Cascade Residual Learning: A Two-stage Convolutional Neural Network for Stereo Matching](https://mlsanity.com/paper/1708.09204/) (2017)
- [Geometry-Aware Symmetric Domain Adaptation for Monocular Depth Estimation](https://mlsanity.com/paper/1904.01870/) (2019)
- [EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth from Light Field Images](https://mlsanity.com/paper/1804.02379/) (2018)
- [Deep Depth From Focus](https://mlsanity.com/paper/1704.01085/) (2017)
- [A 4D Light-Field Dataset and CNN Architectures for Material Recognition](https://mlsanity.com/paper/1608.06985/) (2016)
- [Aperture Supervision for Monocular Depth Estimation](https://mlsanity.com/paper/1711.07933/) (2017)
- [Attention-based Context Aggregation Network for Monocular Depth Estimation](https://mlsanity.com/paper/1901.10137/) (2019)
- [Variational Depth from Focus Reconstruction](https://mlsanity.com/paper/1408.0173/) (2014)
- [Temporally Consistent Depth Prediction with Flow-Guided Memory Units](https://mlsanity.com/paper/1909.07074/) (2019)
- [Towards High Performance Video Object Detection for Mobiles](https://mlsanity.com/paper/1804.05830/) (2018)
- [The Application of Preconditioned Alternating Direction Method of Multipliers in Depth from Focal Stack](https://mlsanity.com/paper/1711.07721/) (2017)

## Built on by (most cited first, 1 of 1)

- [RGB-D-Fusion: Image Conditioned Depth Diffusion of Humanoid Subjects](https://mlsanity.com/paper/2307.15988/) (2023)

## Abstract (arXiv)

Focus based methods have shown promising results for the task of depth estimation. However, most existing focus based depth estimation approaches depend on maximal sharpness of the focal stack. Out of focus information in the focal stack poses challenges for this task. In this paper, we propose a dynamically multi modal learning strategy which incorporates RGB data and the focal stack in our framework. Our goal is to deeply excavate the spatial correlation in the focal stack by designing the spatial correlation perception module and dynamically fuse multi modal information between RGB data and the focal stack in a adaptive way by designing the multi modal dynamic fusion module. The success of our method is demonstrated by achieving the state of the art performance on two datasets. Furthermore, we test our network on a set of different focused images generated by a smart phone camera to prove that the proposed method not only broke the limitation of only using light field data, but also open a path toward practical applications of depth estimation on common consumer level cameras data.