Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs
Not on the map. Why?
Abstract (arXiv)
We present an end-to-end neural network-based model for inferring an approximate 3D mesh representation of a human face from single camera input for AR applications. The relatively dense mesh model of 468 vertices is well-suited for face-based AR effects. The proposed model demonstrates super-realtime inference speed on mobile GPUs (100-1000+ FPS, depending on the device and model variant) and a high prediction quality that is comparable to the variance in manual annotations of the same image.
Key facts. Cited by 73 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Jul 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 4; 2022: 8; 2023: 16; 2024: 14; 2025: 8; 2026: 23.
73 papers.
Sort:
By area:
- 2022Generating Holistic 3D Human Motion from Speechcited by 180
- 2022Towards Metrical Reconstruction of Human Facescited by 166
- 20223D face reconstruction with dense landmarkscited by 70
- 2021LipSync3D: Data-Efficient Learning of Personalized 3D Talking Faces from Video using Pose and Lighting Normalizationcited by 67
- 2023HeadSculpt: Crafting 3D Head Avatars with Textcited by 49
- 20233D Face Reconstruction with the Geometric Guidance of Facial Part Segmentationcited by 38
- 2023SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and Benchmarkcited by 29
- 2023Benchmarking Algorithmic Bias in Face Recognition: An Experimental Approach Using Synthetic Faces and Human Evaluationcited by 25
- 2023LibreFace: An Open-Source Toolkit for Deep Facial Expression Analysiscited by 18
- 2024DSL-FIQA: Assessing Facial Image Quality via Dual-Set Degradation Learning and Landmark-Guided Transformercited by 17
- 2021Real-time RGBD-based Extended Body Pose Estimationcited by 12
- 2023GANonymization: A GAN-based Face Anonymization Framework for Preserving Emotional Expressionscited by 9
38 more
- 2025PhysDrive: A Multimodal Remote Physiological Measurement Dataset for In-vehicle Driver Monitoringcited by 7
- 2021Unsupervised High-Fidelity Facial Texture Generation and Reconstructioncited by 6
- 2024MSPM: A Multi-Site Physiological Monitoring Dataset for Remote Pulse, Respiration, and Blood Pressure Estimationcited by 4
- 2026Universal computational thermal imaging overcoming the ghosting effectcited by 4
- 2022Intelligent Sight and Sound: A Chronic Cancer Pain Datasetcited by 3
- 2024PhysFlow: Skin tone transfer for remote heart rate estimation through conditional normalizing flowscited by 3
- 2025Is It Really You? Exploring Biometric Verification Scenarios in Photorealistic Talking-Head Avatar Videoscited by 3
- 2023Real-time Driver Monitoring Systems on Edge AI Devicecited by 2
- 2024NeRF-AD: Neural Radiance Field with Attention-based Disentanglement for Talking Face Synthesiscited by 2
- 2024Deep adaptative spectral zoom for improved remote heart rate estimationcited by 2
- 2024Evaluation of Video-Based rPPG in Challenging Environments: Artifact Mitigation and Network Resiliencecited by 2
- 2024Learning Gaze-aware Compositional GANcited by 2
- 2024Open-Source Periorbital Segmentation Dataset for Ophthalmic Applicationscited by 2
- 2025A Bridge from Audio to Video: Phoneme-Viseme Alignment Allows Every Face to Speak Multiple Languagescited by 2
- 2026SemanticFace: Semantic Facial Action Estimation via Semantic Distillation in Interpretable Spacecited by 2
- 2022Hiding Your Signals: A Security Analysis of PPG-based Biometric Authenticationcited by 1
- 2023Vista-Morph: Unsupervised Image Registration of Visible-Thermal Facial Pairscited by 1
- 2023A lightweight 3D dense facial landmark estimation model from position map datacited by 1
- 2023GAME: Generalized deep learning model towards multimodal data integration for early screening of adolescent mental disorderscited by 1
- 2023Classification of Safety Driver Attention During Autonomous Vehicle Operationcited by 1
- 2025Measuring Anxiety Levels with Head Motion Patterns in Severe Depression Populationcited by 1
- 2025SkinMap: Weighted Full-Body Skin Segmentation for Robust Remote Photoplethysmographycited by 1
- 2026Emotion-Conditioned Short-Horizon Human Pose Forecasting with a Lightweight Predictive World Modelcited by 1
- 2026Envisage: Diffusion-Based Rhinoplasty Goal Visualization with Mask-Decomposed Evaluationcited by 1
- 2026Geometry-Driven Shadow Harmonisation for Composited Faces: A Multiplicative, Albedo-Preserving Relighting Pipelinecited by 1
- 2021FaceAtlasAR: Atlas of Facial Acupuncture Points in Augmented Realitycited by 0
- 2022Fast Eye Detector Using Siamese Network for NIR Partial Face Imagescited by 0
- 2022Focus Plus: Detect Learner's Distraction by Web Camera in Distance Teachingcited by 0
- 2022SleepyWheels: An Ensemble Model for Drowsiness Detection leading to Accident Preventioncited by 0
- 2023Medical Face Masks and Emotion Recognition from the Body: Insights from a Deep Learning Perspectivecited by 0
- 2023Virtual Avatar Stream: a cost-down approach to the Metaverse experiencecited by 0
- 2023Multi-Domain Norm-referenced Encoding Enables Data Efficient Transfer Learning of Facial Expression Recognitioncited by 0
- 2023Facial Reenactment Through a Personalized Generatorcited by 0
- 2023Portrait Stylization: Artistic Style Transfer with Auxiliary Networks for Human Face Stylizationcited by 0
- 2024JeFaPaTo -- A joint toolbox for blinking analysis and facial features extractioncited by 0
- 2024Examining Monitoring System: Detecting Abnormal Behavior In Online Examinationscited by 0
- 2024Deep Pulse-Signal Magnification for remote Heart Rate Estimation in Compressed Videoscited by 0
- 2024DISCOVER: A Data-driven Interactive System for Comprehensive Observation, Visualization, and ExploRation of Human Behaviourcited by 0
50 most cited of 73 All 73 (JSON)
- 2026FaceKit: a Toolkit for Interpretable Facial Phenotyping, Synthetic Image Generation and Privacy Analysis in Rare Diseases7 Oct 2026
- 2026SeetaPsych v1.0: An Open-source Computer Vision Toolkit for Behavior-based Psychological Measurement17 Sep 2026
- 2026Geometry-Driven Shadow Harmonisation for Composited Faces: A Multiplicative, Albedo-Preserving Relighting Pipeline15 Sep 2026
- 2026TopoRig: Topology-Agnostic Facial Rigging via Multi-Source Supervision14 Sep 2026
- 2026MultiHuSE: A Multimodal Dataset for Humour Styles and Emotions10 Sep 2026
- 2026Emo-DVS: A Multimodal Benchmark for Privacy-Aware Emotion Recognition with Event Cameras7 Sep 2026
- 2026Unwarping the Lens: A Physics-Grounded Approach to Video Glasses Removal20 Aug 2026
- 2026Surface-to-Skeleton 3D Cephalometry: Estimating Hidden Skeletal Landmarks from CT-Derived External Soft-Tissue Surfaces12 Aug 2026
- 2026Measuring Browser Webcam Gaze Honestly: A Capture-Clock Methodology and Open Reference Implementation12 Aug 2026
- 2026Label-Free Parkinson's Disease Screening from Face and Voice through Mechanistic Interpretability10 Aug 2026
- 2026TaoMate: Anchor-Guided Memory Bridging Evolving and Reference States for Real-Time Audio-Video Digital Human Generation27 Jul 2026
- 2026Face Age Verification Vulnerabilities Under Simple Appearance Manipulations27 Jul 2026
38 more
- 2026MagicMakeup: A Region-Controllable Diffusion Transformer for High-Fidelity Makeup-Transfer23 Jul 2026
- 2026Video-Based Detection of squint and cataract for accessibility-aware adaptive web interface rendering8 Jul 2026
- 2026Envisage: Diffusion-Based Rhinoplasty Goal Visualization with Mask-Decomposed Evaluation26 Jun 2026
- 2026Artificial Intelligence can Recognize Whether a Job Applicant is Selling and/or Lying According to Facial Expressions and Head Movements Much More Correctly Than Human Interviewers17 May 2026
- 2026Face inpainting with Identity Preserving Latent Diffusion Models15 May 2026
- 2026Emotion-Conditioned Short-Horizon Human Pose Forecasting with a Lightweight Predictive World Model26 Apr 2026
- 2026Universal computational thermal imaging overcoming the ghosting effect2 Apr 2026
- 2026SemanticFace: Semantic Facial Action Estimation via Semantic Distillation in Interpretable Space16 Mar 2026
- 2026Deployment-Oriented Session-wise Meta-Calibration for Landmark-Based Webcam Gaze Tracking12 Mar 2026
- 2026SKINOPATHY AI: Smartphone-Based Ophthalmic Screening and Longitudinal Tracking Using Lightweight Computer Vision26 Feb 2026
- 2026Markerless Head Tracking for Accurate and Accessible Neuronavigation4 Feb 2026
- 2025A Bridge from Audio to Video: Phoneme-Viseme Alignment Allows Every Face to Speak Multiple Languages8 Oct 2025
- 2025Ocular-Induced Abnormal Head Posture: Diagnosis and Missing Data Imputation7 Oct 2025
- 2025SkinMap: Weighted Full-Body Skin Segmentation for Robust Remote Photoplethysmography6 Oct 2025
- 2025Is It Really You? Exploring Biometric Verification Scenarios in Photorealistic Talking-Head Avatar Videos1 Aug 2025
- 2025PhysDrive: A Multimodal Remote Physiological Measurement Dataset for In-vehicle Driver Monitoring25 Jul 2025
- 2025FairDeFace: Evaluating the Fairness and Adversarial Robustness of Face Obfuscation Methods11 Mar 2025
- 2025AffectSRNet : Facial Emotion-Aware Super-Resolution Network14 Feb 2025
- 2025Measuring Anxiety Levels with Head Motion Patterns in Severe Depression Population12 Feb 2025
- 2024VTD: Visual and Tactile Database for Driver State and Behavior Perception6 Dec 2024
- 2024Open-Source Periorbital Segmentation Dataset for Ophthalmic Applications30 Sep 2024
- 2024State-of-the-Art Periorbital Distance Prediction and Disease Classification Using Periorbital Features27 Sep 2024
- 2024PhysFlow: Skin tone transfer for remote heart rate estimation through conditional normalizing flows31 Jul 2024
- 2024DISCOVER: A Data-driven Interactive System for Comprehensive Observation, Visualization, and ExploRation of Human Behaviour18 Jul 2024
- 2024DSL-FIQA: Assessing Facial Image Quality via Dual-Set Degradation Learning and Landmark-Guided Transformer13 Jun 2024
- 2024Learning Gaze-aware Compositional GAN31 May 2024
- 2024Deep Pulse-Signal Magnification for remote Heart Rate Estimation in Compressed Videos4 May 2024
- 2024Evaluation of Video-Based rPPG in Challenging Environments: Artifact Mitigation and Network Resilience2 May 2024
- 2024Deep adaptative spectral zoom for improved remote heart rate estimation11 Mar 2024
- 2024Examining Monitoring System: Detecting Abnormal Behavior In Online Examinations19 Feb 2024
- 2024JeFaPaTo -- A joint toolbox for blinking analysis and facial features extraction13 Feb 2024
- 2024MSPM: A Multi-Site Physiological Monitoring Dataset for Remote Pulse, Respiration, and Blood Pressure Estimation3 Feb 2024
- 2024NeRF-AD: Neural Radiance Field with Attention-based Disentanglement for Talking Face Synthesis23 Jan 2024
- 20233D Face Reconstruction with the Geometric Guidance of Facial Part Segmentation1 Dec 2023
- 2023SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and Benchmark31 Oct 2023
- 2023Classification of Safety Driver Attention During Autonomous Vehicle Operation17 Oct 2023
- 2023Portrait Stylization: Artistic Style Transfer with Auxiliary Networks for Human Face Stylization23 Sep 2023
- 2023GAME: Generalized deep learning model towards multimodal data integration for early screening of adolescent mental disorders18 Sep 2023
50 newest of 73 All 73 (JSON)
- 2022Generating Holistic 3D Human Motion from Speechcited by 180
- 2022Towards Metrical Reconstruction of Human Facescited by 166
- 20223D face reconstruction with dense landmarkscited by 70
- 2021LipSync3D: Data-Efficient Learning of Personalized 3D Talking Faces from Video using Pose and Lighting Normalizationcited by 67
- 20233D Face Reconstruction with the Geometric Guidance of Facial Part Segmentationcited by 38
- 2023SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and Benchmarkcited by 29
- 2023Benchmarking Algorithmic Bias in Face Recognition: An Experimental Approach Using Synthetic Faces and Human Evaluationcited by 25
- 2023LibreFace: An Open-Source Toolkit for Deep Facial Expression Analysiscited by 18
- 2024DSL-FIQA: Assessing Facial Image Quality via Dual-Set Degradation Learning and Landmark-Guided Transformercited by 17
- 2021Real-time RGBD-based Extended Body Pose Estimationcited by 12
- 2023GANonymization: A GAN-based Face Anonymization Framework for Preserving Emotional Expressionscited by 9
- 2025PhysDrive: A Multimodal Remote Physiological Measurement Dataset for In-vehicle Driver Monitoringcited by 7
12 most cited of 67 All 67 (JSON)
- 2026FaceKit: a Toolkit for Interpretable Facial Phenotyping, Synthetic Image Generation and Privacy Analysis in Rare Diseases7 Oct 2026
- 2026SeetaPsych v1.0: An Open-source Computer Vision Toolkit for Behavior-based Psychological Measurement17 Sep 2026
- 2026Geometry-Driven Shadow Harmonisation for Composited Faces: A Multiplicative, Albedo-Preserving Relighting Pipeline15 Sep 2026
- 2026TopoRig: Topology-Agnostic Facial Rigging via Multi-Source Supervision14 Sep 2026
- 2026MultiHuSE: A Multimodal Dataset for Humour Styles and Emotions10 Sep 2026
- 2026Emo-DVS: A Multimodal Benchmark for Privacy-Aware Emotion Recognition with Event Cameras7 Sep 2026
- 2026Unwarping the Lens: A Physics-Grounded Approach to Video Glasses Removal20 Aug 2026
- 2026Surface-to-Skeleton 3D Cephalometry: Estimating Hidden Skeletal Landmarks from CT-Derived External Soft-Tissue Surfaces12 Aug 2026
- 2026Measuring Browser Webcam Gaze Honestly: A Capture-Clock Methodology and Open Reference Implementation12 Aug 2026
- 2026TaoMate: Anchor-Guided Memory Bridging Evolving and Reference States for Real-Time Audio-Video Digital Human Generation27 Jul 2026
- 2026Face Age Verification Vulnerabilities Under Simple Appearance Manipulations27 Jul 2026
- 2026Video-Based Detection of squint and cataract for accessibility-aware adaptive web interface rendering8 Jul 2026
12 newest of 67 All 67 (JSON)