# DeepMetricEye: Metric Depth Estimation in Periocular VR Imagery

arXiv: https://arxiv.org/abs/2311.07235 · HTML: https://mlsanity.com/paper/2311.07235/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 65 references that are in the map.
- First posted on arXiv 13 Nov 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation](https://mlsanity.com/paper/2004.08790/) (2020)
- [Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2011.02910/) (2020)
- [U-Net and its variants for medical image segmentation: theory and applications](https://mlsanity.com/paper/2011.01118/) (2020)
- [A Survey on Deep Learning Techniques for Stereo-based Depth Estimation](https://mlsanity.com/paper/2006.02535/) (2020)
- [Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimation](https://mlsanity.com/paper/1803.11029/) (2018)
- [Monocular Depth Estimation Based On Deep Learning: An Overview](https://mlsanity.com/paper/2003.06620/) (2020)
- [DenseNet: Implementing Efficient ConvNet Descriptor Pyramids](https://mlsanity.com/paper/1404.1869/) (2014)
- [Structure-Aware Residual Pyramid Network for Monocular Depth Estimation](https://mlsanity.com/paper/1907.06023/) (2019)
- [RITnet: Real-time Semantic Segmentation of the Eye for Gaze Tracking](https://mlsanity.com/paper/1910.00694/) (2019)
- [TEyeD: Over 20 million real-world eye images with Pupil, Eyelid, and Iris 2D and 3D Segmentations, 2D and 3D Landmarks, 3D Eyeball, Gaze Vector, and Eye Movement Types](https://mlsanity.com/paper/2102.02115/) (2021)
- [OpenEDS: Open Eye Dataset](https://mlsanity.com/paper/1905.03702/) (2019)
- [EllSeg: An Ellipse Segmentation Framework for Robust Gaze Tracking](https://mlsanity.com/paper/2007.09600/) (2020)
- [Pistol: Pupil Invisible Supportive Tool to extract Pupil, Iris, Eye Opening, Eye Movements, Pupil and Iris Gaze Vector, and 2D as well as 3D Gaze](https://mlsanity.com/paper/2201.06799/) (2022)
- [Eyenet: Attention based Convolutional Encoder-Decoder Network for Eye Region Segmentation](https://mlsanity.com/paper/1910.03274/) (2019)
- [A New Periocular Dataset Collected by Mobile Devices in Unconstrained Scenarios](https://mlsanity.com/paper/2011.12427/) (2020)
- [Unconstrained Periocular Recognition: Using Generative Deep Learning Frameworks for Attribute Normalization](https://mlsanity.com/paper/2002.03985/) (2020)
- [A Comparative Analysis of Virtual Reality Head-Mounted Display Systems](https://mlsanity.com/paper/1912.02913/) (2019)

## Abstract (arXiv)

Despite the enhanced realism and immersion provided by VR headsets, users frequently encounter adverse effects such as digital eye strain (DES), dry eye, and potential long-term visual impairment due to excessive eye stimulation from VR displays and pressure from the mask. Recent VR headsets are increasingly equipped with eye-oriented monocular cameras to segment ocular feature maps. Yet, to compute the incident light stimulus and observe periocular condition alterations, it is imperative to transform these relative measurements into metric dimensions. To bridge this gap, we propose a lightweight framework derived from the U-Net 3+ deep learning backbone that we re-optimised, to estimate measurable periocular depth maps. Compatible with any VR headset equipped with an eye-oriented monocular camera, our method reconstructs three-dimensional periocular regions, providing a metric basis for related light stimulus calculation protocols and medical guidelines. Navigating the complexities of data collection, we introduce a Dynamic Periocular Data Generation (DPDG) environment based on UE MetaHuman, which synthesises thousands of training images from a small quantity of human facial scan data. Evaluated on a sample of 36 participants, our method exhibited notable efficacy in the periocular global precision evaluation experiment, and the pupil diameter measurement.