# Dual Pixel Exploration: Simultaneous Depth Estimation and Image Restoration

arXiv: https://arxiv.org/abs/2012.00301 · HTML: https://mlsanity.com/paper/2012.00301/

## Key facts

- Cited by 34 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 1 Dec 2020.

## Built on by (most cited first, 30 of 34)

- [Learning to Deblur using Light Field Generated and Real Defocus Images](https://mlsanity.com/paper/2204.00367/) (2022)
- [Improving Single-Image Defocus Deblurring: How Dual-Pixel Images Help Through Multi-Task Learning](https://mlsanity.com/paper/2108.05251/) (2021)
- [Gaussian Kernel Mixture Network for Single Image Defocus Deblurring](https://mlsanity.com/paper/2111.00454/) (2021)
- [Stereo Hybrid Event-Frame (SHEF) Cameras for 3D Perception](https://mlsanity.com/paper/2110.04988/) (2021)
- [DC2: Dual-Camera Defocus Control by Learning to Refocus](https://mlsanity.com/paper/2304.03285/) (2023)
- [SYNAuG: Exploiting Synthetic Data for Data Imbalance Problems](https://mlsanity.com/paper/2308.00994/) (2023)
- [Facial Depth and Normal Estimation using Single Dual-Pixel Camera](https://mlsanity.com/paper/2111.12928/) (2021)
- [LDP: Language-driven Dual-Pixel Image Defocus Deblurring Network](https://mlsanity.com/paper/2307.09815/) (2023)
- [Image Deblurring by Exploring In-depth Properties of Transformer](https://mlsanity.com/paper/2303.15198/) (2023)
- [Learning Depth from Focus in the Wild](https://mlsanity.com/paper/2207.09658/) (2022)
- [Dual-Pixel Raindrop Removal](https://mlsanity.com/paper/2210.13321/) (2022)
- [Learning Single Image Defocus Deblurring with Misaligned Training Pairs](https://mlsanity.com/paper/2211.14502/) (2022)
- [Passive Snapshot Coded Aperture Dual-Pixel RGB-D Imaging](https://mlsanity.com/paper/2402.18102/) (2024)
- [Dynamic Neural Radiance Field From Defocused Monocular Video](https://mlsanity.com/paper/2407.05586/) (2024)
- [Learning Dual-Pixel Alignment for Defocus Deblurring](https://mlsanity.com/paper/2204.12105/) (2022)
- [Point-and-Shoot All-in-Focus Photo Synthesis from Smartphone Camera Pair](https://mlsanity.com/paper/2304.04917/) (2023)
- [Continuous Cost Aggregation for Dual-Pixel Disparity Extraction](https://mlsanity.com/paper/2306.07921/) (2023)
- [Repurposing Marigold for Zero-Shot Metric Depth Estimation via Defocus Blur Cues](https://mlsanity.com/paper/2505.17358/) (2025)
- [DMTNet: Dynamic Multi-scale Network for Dual-pixel Images Defocus Deblurring with Transformer](https://mlsanity.com/paper/2209.06040/) (2022)
- [BaMBNet: A Blur-aware Multi-branch Network for Defocus Deblurring](https://mlsanity.com/paper/2105.14766/) (2021)
- [Learnable Blur Kernel for Single-Image Defocus Deblurring in the Wild](https://mlsanity.com/paper/2211.14017/) (2022)
- [EV-MGDispNet: Motion-Guided Event-Based Stereo Disparity Estimation Network with Left-Right Consistency](https://mlsanity.com/paper/2408.05452/) (2024)
- [Disparity Estimation Using a Quad-Pixel Sensor](https://mlsanity.com/paper/2409.00665/) (2024)
- [Simulating Dual-Pixel Images From Ray Tracing For Depth Estimation](https://mlsanity.com/paper/2503.11213/) (2025)
- [Dynamic Gaussian Splatting from Defocused and Motion-blurred Monocular Videos](https://mlsanity.com/paper/2510.10691/) (2025)
- [Stereo-Knowledge Distillation from dpMV to Dual Pixels for Light Field Video Reconstruction](https://mlsanity.com/paper/2405.11823/) (2024)
- [Revisiting Disparity from Dual-Pixel Images: Physics-Informed Lightweight Depth Estimation](https://mlsanity.com/paper/2411.04714/) (2024)
- [Physically Grounded Monocular Depth via Nanophotonic Wavefront Encoding](https://mlsanity.com/paper/2503.15770/) (2025)
- [Dual-Camera All-in-Focus Neural Radiance Fields](https://mlsanity.com/paper/2504.16636/) (2025)
- [DiFuse-Net: RGB and Dual-Pixel Depth Estimation using Window Bi-directional Parallax Attention and Cross-modal Transfer Learning](https://mlsanity.com/paper/2506.14709/) (2025)

## Abstract (arXiv)

The dual-pixel (DP) hardware works by splitting each pixel in half and creating an image pair in a single snapshot. Several works estimate depth/inverse depth by treating the DP pair as a stereo pair. However, dual-pixel disparity only occurs in image regions with the defocus blur. The heavy defocus blur in DP pairs affects the performance of matching-based depth estimation approaches. Instead of removing the blur effect blindly, we study the formation of the DP pair which links the blur and the depth information. In this paper, we propose a mathematical DP model which can benefit depth estimation by the blur. These explorations motivate us to propose an end-to-end DDDNet (DP-based Depth and Deblur Network) to jointly estimate the depth and restore the image. Moreover, we define a reblur loss, which reflects the relationship of the DP image formation process with depth information, to regularise our depth estimate in training. To meet the requirement of a large amount of data for learning, we propose the first DP image simulator which allows us to create datasets with DP pairs from any existing RGBD dataset. As a side contribution, we collect a real dataset for further research. Extensive experimental evaluation on both synthetic and real datasets shows that our approach achieves competitive performance compared to state-of-the-art approaches.