# Facial Video-based Remote Physiological Measurement via Self-supervised Learning

arXiv: https://arxiv.org/abs/2210.15401 · HTML: https://mlsanity.com/paper/2210.15401/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 73 references that are in the map.
- First posted on arXiv 27 Oct 2022.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Understanding the Behaviour of Contrastive Loss](https://mlsanity.com/paper/2012.09740/) (2020)
- [Spatiotemporal Contrastive Video Representation Learning](https://mlsanity.com/paper/2008.03800/) (2020)
- [Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models](https://mlsanity.com/paper/1911.03393/) (2019)
- [VideoMoCo: Contrastive Video Representation Learning with Temporally Adversarial Examples](https://mlsanity.com/paper/2103.05905/) (2021)
- [Representation Learning by Learning to Count](https://mlsanity.com/paper/1708.06734/) (2017)
- [PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer](https://mlsanity.com/paper/2111.12082/) (2021)
- [BMBC:Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolation](https://mlsanity.com/paper/2007.12622/) (2020)
- [RhythmNet: End-to-end Heart Rate Estimation from Face via Spatial-temporal Representation](https://mlsanity.com/paper/1910.11515/) (2019)
- [Asymmetric Bilateral Motion Estimation for Video Frame Interpolation](https://mlsanity.com/paper/2108.06815/) (2021)
- [Contrastive Learning with Stronger Augmentations](https://mlsanity.com/paper/2104.07713/) (2021)
- [Remote Heart Rate Measurement from Highly Compressed Facial Videos: an End-to-end Deep Learning Solution with Video Enhancement](https://mlsanity.com/paper/1907.11921/) (2019)
- [Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling](https://mlsanity.com/paper/2007.08213/) (2020)
- [Equivariant Contrastive Learning](https://mlsanity.com/paper/2111.00899/) (2021)
- [The Way to my Heart is through Contrastive Learning: Remote Photoplethysmography from Unlabelled Video](https://mlsanity.com/paper/2111.09748/) (2021)
- [PulseGAN: Learning to generate realistic pulse waveforms in remote photoplethysmography](https://mlsanity.com/paper/2006.02699/) (2020)
- [Meta-rPPG: Remote Heart Rate Estimation Using a Transductive Meta-Learner](https://mlsanity.com/paper/2007.06786/) (2020)
- [Camera Measurement of Physiological Vital Signs](https://mlsanity.com/paper/2111.11547/) (2021)
- [Self-supervised pre-training and contrastive representation learning for multiple-choice video QA](https://mlsanity.com/paper/2009.08043/) (2020)
- [COMISR: Compression-Informed Video Super-Resolution](https://mlsanity.com/paper/2105.01237/) (2021)
- [A Temporally-Aware Interpolation Network for Video Frame Inpainting](https://mlsanity.com/paper/1803.07218/) (2018)

## Built on by (most cited first, 23 of 23)

- [Contrast-Phys+: Unsupervised and Weakly-supervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast](https://mlsanity.com/paper/2309.06924/) (2023)
- [RhythmFormer: Extracting Patterned rPPG Signals based on Periodic Sparse Attention](https://mlsanity.com/paper/2402.12788/) (2024)
- [Spiking-PhysFormer: Camera-Based Remote Photoplethysmography with Parallel Spike-driven Transformer](https://mlsanity.com/paper/2402.04798/) (2024)
- [egoPPG: Heart Rate Estimation from Eye-Tracking Cameras in Egocentric Systems to Benefit Downstream Vision Tasks](https://mlsanity.com/paper/2502.20879/) (2025)
- [Remote Photoplethysmography in Real-World and Extreme Lighting Scenarios](https://mlsanity.com/paper/2503.11465/) (2025)
- [Self-similarity Prior Distillation for Unsupervised Remote Physiological Measurement](https://mlsanity.com/paper/2311.05100/) (2023)
- [FreqPhys: Repurposing Implicit Physiological Frequency Prior for Robust Remote Photoplethysmography](https://mlsanity.com/paper/2604.00534/) (2026)
- [SFDA-rPPG: Source-Free Domain Adaptive Remote Physiological Measurement with Spatio-Temporal Consistency](https://mlsanity.com/paper/2409.12040/) (2024)
- [Toward Motion Robustness: A masked attention regularization framework in remote photoplethysmography](https://mlsanity.com/paper/2407.06653/) (2024)
- [DD-rPPGNet: De-interfering and Descriptive Feature Learning for Unsupervised rPPG Estimation](https://mlsanity.com/paper/2407.21402/) (2024)
- [Phase-shifted remote photoplethysmography for estimating heart rate and blood pressure from facial video](https://mlsanity.com/paper/2401.04560/) (2024)
- [SiNC+: Adaptive Camera-Based Vitals with Unsupervised Learning of Periodic Signals](https://mlsanity.com/paper/2404.13449/) (2024)
- [Recovering Pulse Waves from Video Using Deep Unrolling and Deep Equilibrium Models](https://mlsanity.com/paper/2503.17269/) (2025)
- [Periodic-MAE: Periodic Video Masked Autoencoder for rPPG Estimation](https://mlsanity.com/paper/2506.21855/) (2025)
- [rPPG-VQA: A Video Quality Assessment Framework for Unsupervised rPPG Training](https://mlsanity.com/paper/2604.11156/) (2026)
- [Biometric Authentication Based on Enhanced Remote Photoplethysmography Signal Morphology](https://mlsanity.com/paper/2407.04127/) (2024)
- [Bootstrapping Vision-language Models for Self-supervised Remote Physiological Measurement](https://mlsanity.com/paper/2407.08507/) (2024)
- [Time-Series U-Net with Recurrence for Noise-Robust Imaging Photoplethysmography](https://mlsanity.com/paper/2503.17351/) (2025)
- [Non-Contact Physiological Monitoring in Pediatric Intensive Care Units via Adaptive Masking and Self-Supervised Learning](https://mlsanity.com/paper/2602.15967/) (2026)
- [Reliability-Aware Weighted Multi-Scale Spatio-Temporal Maps for Heart Rate Monitoring](https://mlsanity.com/paper/2603.26836/) (2026)
- [Uncertainty-quantified Pulse Signal Recovery from Facial Video using Regularized Stochastic Interpolants](https://mlsanity.com/paper/2604.10777/) (2026)
- [FCUS-rPPG: A Fast-Converging Unsupervised Framework for Remote Photoplethysmography via Gradient Oscillation Suppression](https://mlsanity.com/paper/2606.03050/) (2026)
- [Learning to Assess Heartbeat Observability for mmWave Heart-Rate Sensing](https://mlsanity.com/paper/2610.03570/) (2026)

## Abstract (arXiv)

Facial video-based remote physiological measurement aims to estimate remote photoplethysmography (rPPG) signals from human face videos and then measure multiple vital signs (e.g. heart rate, respiration frequency) from rPPG signals. Recent approaches achieve it by training deep neural networks, which normally require abundant facial videos and synchronously recorded photoplethysmography (PPG) signals for supervision. However, the collection of these annotated corpora is not easy in practice. In this paper, we introduce a novel frequency-inspired self-supervised framework that learns to estimate rPPG signals from facial videos without the need of ground truth PPG signals. Given a video sample, we first augment it into multiple positive/negative samples which contain similar/dissimilar signal frequencies to the original one. Specifically, positive samples are generated using spatial augmentation. Negative samples are generated via a learnable frequency augmentation module, which performs non-linear signal frequency transformation on the input without excessively changing its visual appearance. Next, we introduce a local rPPG expert aggregation module to estimate rPPG signals from augmented samples. It encodes complementary pulsation information from different face regions and aggregate them into one rPPG prediction. Finally, we propose a series of frequency-inspired losses, i.e. frequency contrastive loss, frequency ratio consistency loss, and cross-video frequency agreement loss, for the optimization of estimated rPPG signals from multiple augmented video samples and across temporally neighboring video samples. We conduct rPPG-based heart rate, heart rate variability and respiration frequency estimation on four standard benchmarks. The experimental results demonstrate that our method improves the state of the art by a large margin.