# Facial Depth and Normal Estimation using Single Dual-Pixel Camera arXiv: https://arxiv.org/abs/2111.12928 ยท HTML: https://mlsanity.com/paper/2111.12928/ ## Key facts - Cited by 11 papers in the map (data through 14 Oct 2026). - Builds on 43 of its 82 references that are in the map. - First posted on arXiv 25 Nov 2021. ## Builds on (most cited first, 30 of 43) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016) - [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018) - [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018) - [Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set](https://mlsanity.com/paper/1903.08527/) (2019) - [From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation](https://mlsanity.com/paper/1907.10326/) (2019) - [GA-Net: Guided Aggregation Net for End-to-end Stereo Matching](https://mlsanity.com/paper/1904.06587/) (2019) - [Towards Fast, Accurate and Stable 3D Dense Face Alignment](https://mlsanity.com/paper/2009.09960/) (2020) - [AANet: Adaptive Aggregation Network for Efficient Stereo Matching](https://mlsanity.com/paper/2004.09548/) (2020) - [Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network](https://mlsanity.com/paper/1803.07835/) (2018) - [Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild](https://mlsanity.com/paper/1911.11130/) (2019) - [Nonlinear 3D Face Morphable Model](https://mlsanity.com/paper/1804.03786/) (2018) - [Defocus Deblurring Using Dual-Pixel Data](https://mlsanity.com/paper/2005.00305/) (2020) - [DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image](https://mlsanity.com/paper/1812.00488/) (2018) - [SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wild](https://mlsanity.com/paper/1712.01261/) (2017) - [AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild"](https://mlsanity.com/paper/2003.13845/) (2020) - [StereoNet: Guided Hierarchical Refinement for Real-Time Edge-Aware Depth Prediction](https://mlsanity.com/paper/1807.08865/) (2018) - [Intel RealSense Stereoscopic Depth Cameras](https://mlsanity.com/paper/1705.05548/) (2017) - [Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistency](https://mlsanity.com/paper/2007.12494/) (2020) - [Finding Tiny Faces](https://mlsanity.com/paper/1612.04402/) (2016) - [Learning Detailed Face Reconstruction from a Single Image](https://mlsanity.com/paper/1611.05053/) (2016) - [Synthetic Depth-of-Field with a Single-Camera Mobile Phone](https://mlsanity.com/paper/1806.04171/) (2018) - [Deep Spatial Gradient and Temporal Depth Learning for Face Anti-spoofing](https://mlsanity.com/paper/2003.08061/) (2020) - [Deep Learning for Face Anti-Spoofing: A Survey](https://mlsanity.com/paper/2106.14948/) (2021) - [CASIA-SURF: A Large-scale Multi-modal Benchmark for Face Anti-spoofing](https://mlsanity.com/paper/1908.10654/) (2019) - [Two-shot Spatially-varying BRDF and Shape Estimation](https://mlsanity.com/paper/2004.00403/) (2020) - [A Dataset and Benchmark for Large-scale Multi-modal Face Anti-spoofing](https://mlsanity.com/paper/1812.00408/) (2018) - [Learning Single Camera Depth Estimation using Dual-Pixels](https://mlsanity.com/paper/1904.05822/) (2019) - [Normal Assisted Stereo Depth Estimation](https://mlsanity.com/paper/1911.10444/) (2019) - [MVF-Net: Multi-View 3D Face Morphable Model Regression](https://mlsanity.com/paper/1904.04473/) (2019) ## Built on by (most cited first, 11 of 11) - [Passive Snapshot Coded Aperture Dual-Pixel RGB-D Imaging](https://mlsanity.com/paper/2402.18102/) (2024) - [RetouchingFFHQ: A Large-scale Dataset for Fine-grained Face Retouching Detection](https://mlsanity.com/paper/2307.10642/) (2023) - [Continuous Cost Aggregation for Dual-Pixel Disparity Extraction](https://mlsanity.com/paper/2306.07921/) (2023) - [Learnable Blur Kernel for Single-Image Defocus Deblurring in the Wild](https://mlsanity.com/paper/2211.14017/) (2022) - [IFAST: Weakly Supervised Interpretable Face Anti-spoofing from Single-shot Binocular NIR Images](https://mlsanity.com/paper/2309.17399/) (2023) - [Recent Advancement in 3D Biometrics using Monocular Camera](https://mlsanity.com/paper/2401.02646/) (2024) - [Disparity Estimation Using a Quad-Pixel Sensor](https://mlsanity.com/paper/2409.00665/) (2024) - [Exploring Depth Information for Detecting Manipulated Face Videos](https://mlsanity.com/paper/2411.18572/) (2024) - [Simulating Dual-Pixel Images From Ray Tracing For Depth Estimation](https://mlsanity.com/paper/2503.11213/) (2025) - [Exploring Depth Information for Face Manipulation Detection](https://mlsanity.com/paper/2212.14230/) (2022) - [DP-SfM: Dual-Pixel Structure-from-Motion without Scale Ambiguity](https://mlsanity.com/paper/2605.01852/) (2026) ## Abstract (arXiv) Many mobile manufacturers recently have adopted Dual-Pixel (DP) sensors in their flagship models for faster auto-focus and aesthetic image captures. Despite their advantages, research on their usage for 3D facial understanding has been limited due to the lack of datasets and algorithmic designs that exploit parallax in DP images. This is because the baseline of sub-aperture images is extremely narrow and parallax exists in the defocus blur region. In this paper, we introduce a DP-oriented Depth/Normal network that reconstructs the 3D facial geometry. For this purpose, we collect a DP facial data with more than 135K images for 101 persons captured with our multi-camera structured light systems. It contains the corresponding ground-truth 3D models including depth map and surface normal in metric scale. Our dataset allows the proposed matching network to be generalized for 3D facial depth/normal estimation. The proposed network consists of two novel modules: Adaptive Sampling Module and Adaptive Normal Module, which are specialized in handling the defocus blur in DP images. Finally, the proposed method achieves state-of-the-art performances over recent DP-based depth/normal estimation methods. We also demonstrate the applicability of the estimated depth/normal to face spoofing and relighting.