# Remote Heart Rate Measurement from Highly Compressed Facial Videos: an End-to-end Deep Learning Solution with Video Enhancement

arXiv: https://arxiv.org/abs/1907.11921 · HTML: https://mlsanity.com/paper/1907.11921/

## Key facts

- Cited by 87 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jul 2019.

## Built on by (most cited first, 30 of 87)

- [PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer](https://mlsanity.com/paper/2111.12082/) (2021)
- [The Way to my Heart is through Contrastive Learning: Remote Photoplethysmography from Unlabelled Video](https://mlsanity.com/paper/2111.09748/) (2021)
- [Deep Learning for Face Anti-Spoofing: A Survey](https://mlsanity.com/paper/2106.14948/) (2021)
- [rPPG-Toolbox: Deep Remote PPG Toolbox](https://mlsanity.com/paper/2210.00716/) (2022)
- [Adaptive Normalized Representation Learning for Generalizable Face Anti-Spoofing](https://mlsanity.com/paper/2108.02667/) (2021)
- [Camera Measurement of Physiological Vital Signs](https://mlsanity.com/paper/2111.11547/) (2021)
- [Contrast-Phys: Unsupervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast](https://mlsanity.com/paper/2208.04378/) (2022)
- [PhysFormer++: Facial Video-based Physiological Measurement with SlowFast Temporal Difference Transformer](https://mlsanity.com/paper/2302.03548/) (2023)
- [TransRPPG: Remote Photoplethysmography Transformer for 3D Mask Face Presentation Attack Detection](https://mlsanity.com/paper/2104.07419/) (2021)
- [rPPG-MAE: Self-supervised Pre-training with Masked Autoencoders for Remote Physiological Measurement](https://mlsanity.com/paper/2306.02301/) (2023)
- [Contrast-Phys+: Unsupervised and Weakly-supervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast](https://mlsanity.com/paper/2309.06924/) (2023)
- [Neuron Structure Modeling for Generalizable Remote Physiological Measurement](https://mlsanity.com/paper/2303.05955/) (2023)
- [Adaptive Mixture of Experts Learning for Generalizable Face Anti-Spoofing](https://mlsanity.com/paper/2207.09868/) (2022)
- [Self-supervised Representation Learning Framework for Remote Physiological Measurement Using Spatiotemporal Augmentation Loss](https://mlsanity.com/paper/2107.07695/) (2021)
- [Benchmarking Joint Face Spoofing and Forgery Detection with Visual and Physiological Cues](https://mlsanity.com/paper/2208.05401/) (2022)
- [Facial Video-based Remote Physiological Measurement via Self-supervised Learning](https://mlsanity.com/paper/2210.15401/) (2022)
- [Face2PPG: An unsupervised pipeline for blood volume pulse extraction from faces](https://mlsanity.com/paper/2202.04101/) (2022)
- [RhythmMamba: Fast, Lightweight, and Accurate Remote Physiological Measurement](https://mlsanity.com/paper/2404.06483/) (2024)
- [Spiking-PhysFormer: Camera-Based Remote Photoplethysmography with Parallel Spike-driven Transformer](https://mlsanity.com/paper/2402.04798/) (2024)
- [Digital and Physical Face Attacks: Reviewing and One Step Further](https://mlsanity.com/paper/2209.14692/) (2022)
- [PhysMamba: Efficient Remote Physiological Measurement with SlowFast Temporal Difference Mamba](https://mlsanity.com/paper/2409.12031/) (2024)
- [Efficient Remote Photoplethysmography with Temporal Derivative Modules and Time-Shift Invariant Loss](https://mlsanity.com/paper/2203.10882/) (2022)
- [Motion Matters: Neural Motion Transfer for Better Camera Physiological Measurement](https://mlsanity.com/paper/2303.12059/) (2023)
- [Depression Recognition using Remote Photoplethysmography from Facial Videos](https://mlsanity.com/paper/2206.04399/) (2022)
- [PhysMLE: Generalizable and Priors-Inclusive Multi-task Remote Physiological Measurement](https://mlsanity.com/paper/2405.06201/) (2024)
- [Camera-Based HRV Prediction for Remote Learning Environments](https://mlsanity.com/paper/2305.04161/) (2023)
- [How Suboptimal is Training rPPG Models with Videos and Targets from Different Body Sites?](https://mlsanity.com/paper/2403.10582/) (2024)
- [TransPPG: Two-stream Transformer for Remote Heart Rate Estimate](https://mlsanity.com/paper/2201.10873/) (2022)
- [La-SoftMoE CLIP for Unified Physical-Digital Face Attack Detection](https://mlsanity.com/paper/2408.12793/) (2024)
- [Overcoming Difficulty in Obtaining Dark-skinned Subjects for Remote-PPG by Synthetic Augmentation](https://mlsanity.com/paper/2106.06007/) (2021)

## Abstract (arXiv)

Remote photoplethysmography (rPPG), which aims at measuring heart activities without any contact, has great potential in many applications (e.g., remote healthcare). Existing rPPG approaches rely on analyzing very fine details of facial videos, which are prone to be affected by video compression. Here we propose a two-stage, end-to-end method using hidden rPPG information enhancement and attention networks, which is the first attempt to counter video compression loss and recover rPPG signals from highly compressed videos. The method includes two parts: 1) a Spatio-Temporal Video Enhancement Network (STVEN) for video enhancement, and 2) an rPPG network (rPPGNet) for rPPG signal recovery. The rPPGNet can work on its own for robust rPPG measurement, and the STVEN network can be added and jointly trained to further boost the performance especially on highly compressed videos. Comprehensive experiments are performed on two benchmark datasets to show that, 1) the proposed method not only achieves superior performance on compressed videos with high-quality videos pair, 2) it also generalizes well on novel data with only compressed videos available, which implies the promising potential for real world applications.