# Revisiting Disparity from Dual-Pixel Images: Physics-Informed Lightweight Depth Estimation

arXiv: https://arxiv.org/abs/2411.04714 · HTML: https://mlsanity.com/paper/2411.04714/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 59 references that are in the map.
- First posted on arXiv 6 Nov 2024.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Non-Local Spatial Propagation Network for Depth Completion](https://mlsanity.com/paper/2007.10042/) (2020)
- [PENet: Towards Precise and Efficient Image Guided Depth Completion](https://mlsanity.com/paper/2103.00783/) (2021)
- [Defocus Deblurring Using Dual-Pixel Data](https://mlsanity.com/paper/2005.00305/) (2020)
- [Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network](https://mlsanity.com/paper/1808.00150/) (2018)
- [CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completion](https://mlsanity.com/paper/1911.05377/) (2019)
- [The Fast Bilateral Solver](https://mlsanity.com/paper/1511.03296/) (2015)
- [RigNet: Repetitive Image Guided Network for Depth Completion](https://mlsanity.com/paper/2107.13802/) (2021)
- [Synthetic Depth-of-Field with a Single-Camera Mobile Phone](https://mlsanity.com/paper/1806.04171/) (2018)
- [Towards Fast and Accurate Real-World Depth Super-Resolution: Benchmark Dataset and Baseline](https://mlsanity.com/paper/2104.06174/) (2021)
- [Bilateral Propagation Network for Depth Completion](https://mlsanity.com/paper/2403.11270/) (2024)
- [Deep symbolic regression for physics guided by units constraints: toward the automated discovery of physical laws](https://mlsanity.com/paper/2303.03192/) (2023)
- [Learning Single Camera Depth Estimation using Dual-Pixels](https://mlsanity.com/paper/1904.05822/) (2019)
- [LRRU: Long-short Range Recurrent Updating Networks for Depth Completion](https://mlsanity.com/paper/2310.08956/) (2023)
- [Learning to Reduce Defocus Blur by Realistically Modeling Dual-Pixel Data](https://mlsanity.com/paper/2012.03255/) (2020)
- [Dual Pixel Exploration: Simultaneous Depth Estimation and Image Restoration](https://mlsanity.com/paper/2012.00301/) (2020)
- [Learning to Autofocus](https://mlsanity.com/paper/2004.12260/) (2020)
- [Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image](https://mlsanity.com/paper/2110.05655/) (2021)
- [Du2Net: Learning Depth Estimation from Dual-Cameras and Dual-Pixels](https://mlsanity.com/paper/2003.14299/) (2020)
- [SemAttNet: Towards Attention-based Semantic Aware Guided Depth Completion](https://mlsanity.com/paper/2204.13635/) (2022)
- [Improving Single-Image Defocus Deblurring: How Dual-Pixel Images Help Through Multi-Task Learning](https://mlsanity.com/paper/2108.05251/) (2021)
- [Robust Depth Completion with Uncertainty-Driven Loss Functions](https://mlsanity.com/paper/2112.07895/) (2021)
- [Passive Snapshot Coded Aperture Dual-Pixel RGB-D Imaging](https://mlsanity.com/paper/2402.18102/) (2024)

## Abstract (arXiv)

In this study, we propose a high-performance disparity (depth) estimation method using dual-pixel (DP) images with few parameters. Conventional end-to-end deep-learning methods have many parameters but do not fully exploit disparity constraints, which limits their performance. Therefore, we propose a lightweight disparity estimation method based on a completion-based network that explicitly constrains disparity and learns the physical and systemic disparity properties of DP. By modeling the DP-specific disparity error parametrically and using it for sampling during training, the network acquires the unique properties of DP and enhances robustness. This learning also allows us to use a common RGB-D dataset for training without a DP dataset, which is labor-intensive to acquire. Furthermore, we propose a non-learning-based refinement framework that efficiently handles inherent disparity expansion errors by appropriately refining the confidence map of the network output. As a result, the proposed method achieved state-of-the-art results while reducing the overall system size to 1/5 of that of the conventional method, even without using the DP dataset for training, thereby demonstrating its effectiveness. The code and dataset are available on our project site.