# Orientation-conditioned Facial Texture Mapping for Video-based Facial Remote Photoplethysmography Estimation

arXiv: https://arxiv.org/abs/2404.09378 · HTML: https://mlsanity.com/paper/2404.09378/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 14 Apr 2024.

## Builds on (most cited first, 22 of 22)

- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks](https://mlsanity.com/paper/1805.07888/) (2018)
- [Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement](https://mlsanity.com/paper/2006.03790/) (2020)
- [Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks](https://mlsanity.com/paper/1905.02419/) (2019)
- [PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer](https://mlsanity.com/paper/2111.12082/) (2021)
- [RhythmNet: End-to-end Heart Rate Estimation from Face via Spatial-temporal Representation](https://mlsanity.com/paper/1910.11515/) (2019)
- [Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling](https://mlsanity.com/paper/2007.08213/) (2020)
- [EfficientPhys: Enabling Simple, Fast and Accurate Camera-Based Vitals Measurement](https://mlsanity.com/paper/2110.04447/) (2021)
- [rPPG-Toolbox: Deep Remote PPG Toolbox](https://mlsanity.com/paper/2210.00716/) (2022)
- [VIPL-HR: A Multi-modal Database for Pulse Estimation from Less-constrained Face Video](https://mlsanity.com/paper/1810.04927/) (2018)
- [MMPD: Multi-Domain Mobile Video Physiology Dataset](https://mlsanity.com/paper/2302.03840/) (2023)
- [Camera Measurement of Physiological Vital Signs](https://mlsanity.com/paper/2111.11547/) (2021)
- [TransRPPG: Remote Photoplethysmography Transformer for 3D Mask Face Presentation Attack Detection](https://mlsanity.com/paper/2104.07419/) (2021)
- [AutoHR: A Strong End-to-end Baseline for Remote Heart Rate Measurement with Neural Searching](https://mlsanity.com/paper/2004.12292/) (2020)
- [rPPG-MAE: Self-supervised Pre-training with Masked Autoencoders for Remote Physiological Measurement](https://mlsanity.com/paper/2306.02301/) (2023)
- [DeepFakesON-Phys: DeepFakes Detection based on Heart Rate Estimation](https://mlsanity.com/paper/2010.00400/) (2020)
- [Learning Motion-Robust Remote Photoplethysmography through Arbitrary Resolution Videos](https://mlsanity.com/paper/2211.16922/) (2022)
- [Face2PPG: An unsupervised pipeline for blood volume pulse extraction from faces](https://mlsanity.com/paper/2202.04101/) (2022)
- [Dual-path TokenLearner for Remote Photoplethysmography-based Physiological Measurement with Facial Videos](https://mlsanity.com/paper/2308.07771/) (2023)
- [Real-time Webcam Heart-Rate and Variability Estimation with Clean Ground Truth for Evaluation](https://mlsanity.com/paper/2012.15846/) (2020)
- [Motion Matters: Neural Motion Transfer for Better Camera Physiological Measurement](https://mlsanity.com/paper/2303.12059/) (2023)
- [Non-contact PPG Signal and Heart Rate Estimation with Multi-hierarchical Convolutional Network](https://mlsanity.com/paper/2104.02260/) (2021)

## Built on by (most cited first, 6 of 6)

- [Remote Photoplethysmography in Real-World and Extreme Lighting Scenarios](https://mlsanity.com/paper/2503.11465/) (2025)
- [BeatFormer: Efficient motion-robust remote heart rate estimation through unsupervised spectral zoomed attention filters](https://mlsanity.com/paper/2507.14885/) (2025)
- [Facial Expression Recognition with Controlled Privacy Preservation and Feature Compensation](https://mlsanity.com/paper/2412.00277/) (2024)
- [Facial Spatiotemporal Graphs: Leveraging the 3D Facial Surface for Remote Physiological Measurement](https://mlsanity.com/paper/2601.13724/) (2026)
- [Video-based Heart Rate Estimation with Angle-guided ROI Optimization and Graph Signal Denoising](https://mlsanity.com/paper/2604.11395/) (2026)
- [CanonicalPhys: Pose-Robust Remote Photoplethysmography via Canonical-Space Priors](https://mlsanity.com/paper/2607.15995/) (2026)

## Abstract (arXiv)

Camera-based remote photoplethysmography (rPPG) enables contactless measurement of important physiological signals such as pulse rate (PR). However, dynamic and unconstrained subject motion introduces significant variability into the facial appearance in video, confounding the ability of video-based methods to accurately extract the rPPG signal. In this study, we leverage the 3D facial surface to construct a novel orientation-conditioned facial texture video representation which improves the motion robustness of existing video-based facial rPPG estimation methods. Our proposed method achieves a significant 18.2% performance improvement in cross-dataset testing on MMPD over our baseline using the PhysNet model trained on PURE, highlighting the efficacy and generalization benefits of our designed video representation. We demonstrate significant performance improvements of up to 29.6% in all tested motion scenarios in cross-dataset testing on MMPD, even in the presence of dynamic and unconstrained subject motion, emphasizing the benefits of disentangling motion through modeling the 3D facial surface for motion robust facial rPPG estimation. We validate the efficacy of our design decisions and the impact of different video processing steps through an ablation study. Our findings illustrate the potential strengths of exploiting the 3D facial surface as a general strategy for addressing dynamic and unconstrained subject motion in videos. The code is available at https://samcantrill.github.io/orientation-uv-rppg/.