# Fully Self-Supervised Depth Estimation from Defocus Clue

arXiv: https://arxiv.org/abs/2303.10752 · HTML: https://mlsanity.com/paper/2303.10752/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 29 references that are in the map.
- First posted on arXiv 19 Mar 2023.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)
- [GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose](https://mlsanity.com/paper/1803.02276/) (2018)
- [Three Ways to Improve Semantic Segmentation with Self-Supervised Depth Estimation](https://mlsanity.com/paper/2012.10782/) (2020)
- [Cross-Domain Self-supervised Multi-task Feature Learning using Synthetic Imagery](https://mlsanity.com/paper/1711.09082/) (2017)
- [Single Image Depth Estimation Trained via Depth from Defocus Cues](https://mlsanity.com/paper/2001.05036/) (2020)
- [MonoIndoor: Towards Good Practice of Self-Supervised Monocular Depth Estimation for Indoor Environments](https://mlsanity.com/paper/2107.12429/) (2021)
- [Focus on defocus: bridging the synthetic to real domain gap for depth estimation](https://mlsanity.com/paper/2005.09623/) (2020)
- [Bridging Unsupervised and Supervised Depth from Focus via All-in-Focus Supervision](https://mlsanity.com/paper/2108.10843/) (2021)
- [Deep Depth from Focus with Differential Focus Volume](https://mlsanity.com/paper/2112.01712/) (2021)
- [Deep Depth from Defocus: how can defocus blur improve 3D estimation using dense neural networks?](https://mlsanity.com/paper/1809.01567/) (2018)
- [Variational Depth from Focus Reconstruction](https://mlsanity.com/paper/1408.0173/) (2014)
- [Plugging Self-Supervised Monocular Depth into Unsupervised Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2110.06685/) (2021)

## Built on by (most cited first, 17 of 17)

- [Cinematic Gaussians: Real-Time HDR Radiance Fields with Depth of Field](https://mlsanity.com/paper/2406.07329/) (2024)
- [HybridDepth: Robust Metric Depth Fusion by Leveraging Depth from Focus and Single-Image Priors](https://mlsanity.com/paper/2407.18443/) (2024)
- [Hybrid-grained Feature Aggregation with Coarse-to-fine Language Guidance for Self-supervised Monocular Depth Estimation](https://mlsanity.com/paper/2510.09320/) (2025)
- [Blurry-Edges: Photon-Limited Depth Estimation from Defocused Boundaries](https://mlsanity.com/paper/2503.23606/) (2025)
- [Depth Estimation Based on 3D Gaussian Splatting Siamese Defocus](https://mlsanity.com/paper/2409.12323/) (2024)
- [Repurposing Marigold for Zero-Shot Metric Depth Estimation via Defocus Blur Cues](https://mlsanity.com/paper/2505.17358/) (2025)
- [FocDepthFormer: Transformer with latent LSTM for Depth Estimation from Focal Stack](https://mlsanity.com/paper/2310.11178/) (2023)
- [Learning Monocular Depth from Focus with Event Focal Stack](https://mlsanity.com/paper/2405.06944/) (2024)
- [Fovea Stacking: Imaging with Dynamic Localized Aberration Correction](https://mlsanity.com/paper/2506.00716/) (2025)
- [DualFocus: Depth from Focus with Spatio-Focal Dual Variational Constraints](https://mlsanity.com/paper/2509.21992/) (2025)
- [Boosting Monocular Metric Depth Estimation via Bokeh Rendering](https://mlsanity.com/paper/2512.12425/) (2025)
- [Dark Channel-Assisted Depth-from-Defocus from a Single Image](https://mlsanity.com/paper/2506.06643/) (2025)
- [Depth from Defocus via Direct Optimization](https://mlsanity.com/paper/2602.18509/) (2026)
- [Zero-Shot Depth from Defocus](https://mlsanity.com/paper/2603.26658/) (2026)
- [Towards Minimal Focal Stack in Shape from Focus](https://mlsanity.com/paper/2604.01603/) (2026)
- [Depth from Dual Differential Defocus and Stereo Consensus](https://mlsanity.com/paper/2606.02906/) (2026)
- [Monocular Depth Estimation from a Single Image: Progress and Opportunities](https://mlsanity.com/paper/2609.01172/) (2026)

## Abstract (arXiv)

Depth-from-defocus (DFD), modeling the relationship between depth and defocus pattern in images, has demonstrated promising performance in depth estimation. Recently, several self-supervised works try to overcome the difficulties in acquiring accurate depth ground-truth. However, they depend on the all-in-focus (AIF) images, which cannot be captured in real-world scenarios. Such limitation discourages the applications of DFD methods. To tackle this issue, we propose a completely self-supervised framework that estimates depth purely from a sparse focal stack. We show that our framework circumvents the needs for the depth and AIF image ground-truth, and receives superior predictions, thus closing the gap between the theoretical success of DFD works and their applications in the real world. In particular, we propose (i) a more realistic setting for DFD tasks, where no depth or AIF image ground-truth is available; (ii) a novel self-supervision framework that provides reliable predictions of depth and AIF image under the challenging setting. The proposed framework uses a neural model to predict the depth and AIF image, and utilizes an optical model to validate and refine the prediction. We verify our framework on three benchmark datasets with rendered focal stacks and real focal stacks. Qualitative and quantitative evaluations show that our method provides a strong baseline for self-supervised DFD tasks.