# Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image

arXiv: https://arxiv.org/abs/2110.05655 · HTML: https://mlsanity.com/paper/2110.05655/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 96 references that are in the map.
- First posted on arXiv 12 Oct 2021.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)
- [Single-View View Synthesis with Multiplane Images](https://mlsanity.com/paper/2004.11364/) (2020)
- [A General and Adaptive Robust Loss Function](https://mlsanity.com/paper/1701.03077/) (2017)
- [Stereo Magnification: Learning View Synthesis using Multiplane Images](https://mlsanity.com/paper/1805.09817/) (2018)
- [Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints](https://mlsanity.com/paper/1802.05522/) (2018)
- [Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields](https://mlsanity.com/paper/1502.07411/) (2015)
- [DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency](https://mlsanity.com/paper/1809.01649/) (2018)
- [Defocus Deblurring Using Dual-Pixel Data](https://mlsanity.com/paper/2005.00305/) (2020)
- [CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth](https://mlsanity.com/paper/1904.02028/) (2019)
- [Synthetic Depth-of-Field with a Single-Camera Mobile Phone](https://mlsanity.com/paper/1806.04171/) (2018)
- [Learning Monocular Depth by Distilling Cross-domain Stereo Networks](https://mlsanity.com/paper/1808.06586/) (2018)
- [AdaDepth: Unsupervised Content Congruent Adaptation for Depth Estimation](https://mlsanity.com/paper/1803.01599/) (2018)
- [Learning Single Camera Depth Estimation using Dual-Pixels](https://mlsanity.com/paper/1904.05822/) (2019)
- [What Makes Good Synthetic Training Data for Learning Disparity and Optical Flow Estimation?](https://mlsanity.com/paper/1801.06397/) (2018)
- [Deep Depth From Focus](https://mlsanity.com/paper/1704.01085/) (2017)
- [Learning Fully Convolutional Networks for Iterative Non-blind Deconvolution](https://mlsanity.com/paper/1611.06495/) (2016)
- [Learning to Autofocus](https://mlsanity.com/paper/2004.12260/) (2020)
- [A Unified Approach of Multi-scale Deep and Hand-crafted Features for Defocus Estimation](https://mlsanity.com/paper/1704.08992/) (2017)
- [Du2Net: Learning Depth Estimation from Dual-Cameras and Dual-Pixels](https://mlsanity.com/paper/2003.14299/) (2020)
- [Aperture Supervision for Monocular Depth Estimation](https://mlsanity.com/paper/1711.07933/) (2017)
- [Wireless Software Synchronization of Multiple Distributed Cameras](https://mlsanity.com/paper/1812.09366/) (2018)
- [Reversible Jump Metropolis Light Transport using Inverse Mappings](https://mlsanity.com/paper/1704.06835/) (2017)
- [Self-Supervised Relative Depth Learning for Urban Scene Understanding](https://mlsanity.com/paper/1712.04850/) (2017)

## Built on by (most cited first, 25 of 25)

- [DC2: Dual-Camera Defocus Control by Learning to Refocus](https://mlsanity.com/paper/2304.03285/) (2023)
- [Facial Depth and Normal Estimation using Single Dual-Pixel Camera](https://mlsanity.com/paper/2111.12928/) (2021)
- [LDP: Language-driven Dual-Pixel Image Defocus Deblurring Network](https://mlsanity.com/paper/2307.09815/) (2023)
- [Image Deblurring by Exploring In-depth Properties of Transformer](https://mlsanity.com/paper/2303.15198/) (2023)
- [Learning Depth from Focus in the Wild](https://mlsanity.com/paper/2207.09658/) (2022)
- [Learning Single Image Defocus Deblurring with Misaligned Training Pairs](https://mlsanity.com/paper/2211.14502/) (2022)
- [Dual-Pixel Raindrop Removal](https://mlsanity.com/paper/2210.13321/) (2022)
- [Learning Lens Blur Fields](https://mlsanity.com/paper/2310.11535/) (2023)
- [Passive Snapshot Coded Aperture Dual-Pixel RGB-D Imaging](https://mlsanity.com/paper/2402.18102/) (2024)
- [Point-and-Shoot All-in-Focus Photo Synthesis from Smartphone Camera Pair](https://mlsanity.com/paper/2304.04917/) (2023)
- [Efficient Depth- and Spatially-Varying Image Simulation for Defocus Deblur](https://mlsanity.com/paper/2507.00372/) (2025)
- [Continuous Cost Aggregation for Dual-Pixel Disparity Extraction](https://mlsanity.com/paper/2306.07921/) (2023)
- [Efficient Hybrid Zoom using Camera Fusion on Mobile Phones](https://mlsanity.com/paper/2401.01461/) (2024)
- [Repurposing Marigold for Zero-Shot Metric Depth Estimation via Defocus Blur Cues](https://mlsanity.com/paper/2505.17358/) (2025)
- [Learnable Blur Kernel for Single-Image Defocus Deblurring in the Wild](https://mlsanity.com/paper/2211.14017/) (2022)
- [Disparity Estimation Using a Quad-Pixel Sensor](https://mlsanity.com/paper/2409.00665/) (2024)
- [Simulating Dual-Pixel Images From Ray Tracing For Depth Estimation](https://mlsanity.com/paper/2503.11213/) (2025)
- [Fine-grained Defocus Blur Control for Generative Image Models](https://mlsanity.com/paper/2510.06215/) (2025)
- [Learning to Refocus with Video Diffusion Models](https://mlsanity.com/paper/2512.19823/) (2025)
- [Point Spread Function Estimation of Defocus](https://mlsanity.com/paper/2203.02953/) (2022)
- [Stereo-Knowledge Distillation from dpMV to Dual Pixels for Light Field Video Reconstruction](https://mlsanity.com/paper/2405.11823/) (2024)
- [Revisiting Disparity from Dual-Pixel Images: Physics-Informed Lightweight Depth Estimation](https://mlsanity.com/paper/2411.04714/) (2024)
- [Freq-DP Net: A Dual-Branch Network for Fence Removal using Dual-Pixel and Fourier Priors](https://mlsanity.com/paper/2602.14226/) (2026)
- [DP-SfM: Dual-Pixel Structure-from-Motion without Scale Ambiguity](https://mlsanity.com/paper/2605.01852/) (2026)
- [FoundDP: Revisiting Weak Disparity Observability in Dual-Pixel Depth Estimation](https://mlsanity.com/paper/2607.01900/) (2026)

## Abstract (arXiv)

We present a method that takes as input a single dual-pixel image, and simultaneously estimates the image's defocus map -- the amount of defocus blur at each pixel -- and recovers an all-in-focus image. Our method is inspired from recent works that leverage the dual-pixel sensors available in many consumer cameras to assist with autofocus, and use them for recovery of defocus maps or all-in-focus images. These prior works have solved the two recovery problems independently of each other, and often require large labeled datasets for supervised training. By contrast, we show that it is beneficial to treat these two closely-connected problems simultaneously. To this end, we set up an optimization problem that, by carefully modeling the optics of dual-pixel images, jointly solves both problems. We use data captured with a consumer smartphone camera to demonstrate that, after a one-time calibration step, our approach improves upon prior works for both defocus map estimation and blur removal, despite being entirely unsupervised.