# Learning Single Camera Depth Estimation using Dual-Pixels

arXiv: https://arxiv.org/abs/1904.05822 · HTML: https://mlsanity.com/paper/1904.05822/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Apr 2019.

## Built on by (most cited first, 30 of 49)

- [Iterative Filter Adaptive Network for Single Image Defocus Deblurring](https://mlsanity.com/paper/2108.13610/) (2021)
- [Unsupervised Scale-consistent Depth Learning from Video](https://mlsanity.com/paper/2105.11610/) (2021)
- [Self-Supervised Monocular Depth Estimation with Internal Feature Fusion](https://mlsanity.com/paper/2110.09482/) (2021)
- [LightIt: Illumination Modeling and Control for Diffusion Models](https://mlsanity.com/paper/2403.10615/) (2024)
- [Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutions](https://mlsanity.com/paper/2108.09108/) (2021)
- [SLIDE: Single Image 3D Photography with Soft Layering and Depth-aware Inpainting](https://mlsanity.com/paper/2109.01068/) (2021)
- [Guided Co-Modulated GAN for 360deg Field of View Extrapolation](https://mlsanity.com/paper/2204.07286/) (2022)
- [AdvHaze: Adversarial Haze Attack](https://mlsanity.com/paper/2104.13673/) (2021)
- [Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image](https://mlsanity.com/paper/2110.05655/) (2021)
- [Improving Single-Image Defocus Deblurring: How Dual-Pixel Images Help Through Multi-Task Learning](https://mlsanity.com/paper/2108.05251/) (2021)
- [Gaussian Kernel Mixture Network for Single Image Defocus Deblurring](https://mlsanity.com/paper/2111.00454/) (2021)
- [Incremental Joint Learning of Depth, Pose and Implicit Scene Representation on Monocular Camera in Large-scale Scenes](https://mlsanity.com/paper/2404.06050/) (2024)
- [Facial Depth and Normal Estimation using Single Dual-Pixel Camera](https://mlsanity.com/paper/2111.12928/) (2021)
- [A Survey on RGB-D Datasets](https://mlsanity.com/paper/2201.05761/) (2022)
- [LDP: Language-driven Dual-Pixel Image Defocus Deblurring Network](https://mlsanity.com/paper/2307.09815/) (2023)
- [Learning Depth from Focus in the Wild](https://mlsanity.com/paper/2207.09658/) (2022)
- [SUB-Depth: Self-distillation and Uncertainty Boosting Self-supervised Monocular Depth Estimation](https://mlsanity.com/paper/2111.09692/) (2021)
- [Deep Depth from Focal Stack with Defocus Model for Camera-Setting Invariance](https://mlsanity.com/paper/2202.13055/) (2022)
- [Towards Domain-agnostic Depth Completion](https://mlsanity.com/paper/2207.14466/) (2022)
- [Dual-Pixel Raindrop Removal](https://mlsanity.com/paper/2210.13321/) (2022)
- [Passive Snapshot Coded Aperture Dual-Pixel RGB-D Imaging](https://mlsanity.com/paper/2402.18102/) (2024)
- [Learning Dual-Pixel Alignment for Defocus Deblurring](https://mlsanity.com/paper/2204.12105/) (2022)
- [Learned Monocular Depth Priors in Visual-Inertial Initialization](https://mlsanity.com/paper/2204.09171/) (2022)
- [Multi-View Motion Synthesis via Applying Rotated Dual-Pixel Blur Kernels](https://mlsanity.com/paper/2111.07837/) (2021)
- [Continuous Cost Aggregation for Dual-Pixel Disparity Extraction](https://mlsanity.com/paper/2306.07921/) (2023)
- [Repurposing Marigold for Zero-Shot Metric Depth Estimation via Defocus Blur Cues](https://mlsanity.com/paper/2505.17358/) (2025)
- [Promoting CNNs with Cross-Architecture Knowledge Distillation for Efficient Monocular Depth Estimation](https://mlsanity.com/paper/2404.16386/) (2024)
- [SaccadeCam: Adaptive Visual Attention for Monocular Depth Sensing](https://mlsanity.com/paper/2103.12981/) (2021)
- [BaMBNet: A Blur-aware Multi-branch Network for Defocus Deblurring](https://mlsanity.com/paper/2105.14766/) (2021)
- [DASHA: Decentralized Autofocusing System with Hierarchical Agents](https://mlsanity.com/paper/2108.12842/) (2021)

## Abstract (arXiv)

Deep learning techniques have enabled rapid progress in monocular depth estimation, but their quality is limited by the ill-posed nature of the problem and the scarcity of high quality datasets. We estimate depth from a single camera by leveraging the dual-pixel auto-focus hardware that is increasingly common on modern camera sensors. Classic stereo algorithms and prior learning-based depth estimation techniques under-perform when applied on this dual-pixel data, the former due to too-strong assumptions about RGB image matching, and the latter due to not leveraging the understanding of optics of dual-pixel image formation. To allow learning based methods to work well on dual-pixel imagery, we identify an inherent ambiguity in the depth estimated from dual-pixel cues, and develop an approach to estimate depth up to this ambiguity. Using our approach, existing monocular depth estimation techniques can be effectively applied to dual-pixel data, and much smaller models can be constructed that still infer high quality depth. To demonstrate this, we capture a large dataset of in-the-wild 5-viewpoint RGB images paired with corresponding dual-pixel data, and show how view supervision with this data can be used to learn depth up to the unknown ambiguities. On our new task, our model is 30% more accurate than any prior work on learning-based monocular or stereoscopic depth estimation.