# Non-contact PPG Signal and Heart Rate Estimation with Multi-hierarchical Convolutional Network

arXiv: https://arxiv.org/abs/2104.02260 · HTML: https://mlsanity.com/paper/2104.02260/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 40 references that are in the map.
- First posted on arXiv 6 Apr 2021.

## Builds on (most cited first, 13 of 13)

- [Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks](https://mlsanity.com/paper/1711.10305/) (2017)
- [DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks](https://mlsanity.com/paper/1805.07888/) (2018)
- [Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement](https://mlsanity.com/paper/2006.03790/) (2020)
- [Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks](https://mlsanity.com/paper/1905.02419/) (2019)
- [RhythmNet: End-to-end Heart Rate Estimation from Face via Spatial-temporal Representation](https://mlsanity.com/paper/1910.11515/) (2019)
- [Remote Heart Rate Measurement from Highly Compressed Facial Videos: an End-to-end Deep Learning Solution with Video Enhancement](https://mlsanity.com/paper/1907.11921/) (2019)
- [Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling](https://mlsanity.com/paper/2007.08213/) (2020)
- [PulseGAN: Learning to generate realistic pulse waveforms in remote photoplethysmography](https://mlsanity.com/paper/2006.02699/) (2020)
- [Meta-rPPG: Remote Heart Rate Estimation Using a Transductive Meta-Learner](https://mlsanity.com/paper/2007.06786/) (2020)
- [A Reproducible Study on Remote Heart Rate Measurement](https://mlsanity.com/paper/1709.00962/) (2017)
- [AutoHR: A Strong End-to-end Baseline for Remote Heart Rate Measurement with Neural Searching](https://mlsanity.com/paper/2004.12292/) (2020)
- [DistancePPG: Robust non-contact vital signs monitoring using a camera](https://mlsanity.com/paper/1502.08040/) (2015)
- [Deep-HR: Fast Heart Rate Estimation from Face Video Under Realistic Conditions](https://mlsanity.com/paper/2002.04821/) (2020)

## Built on by (most cited first, 5 of 5)

- [Orientation-conditioned Facial Texture Mapping for Video-based Facial Remote Photoplethysmography Estimation](https://mlsanity.com/paper/2404.09378/) (2024)
- [Fully Test-Time rPPG Estimation via Synthetic Signal-Guided Feature Learning](https://mlsanity.com/paper/2407.13322/) (2024)
- [Deep adaptative spectral zoom for improved remote heart rate estimation](https://mlsanity.com/paper/2403.06902/) (2024)
- [An AI-Enabled Framework Within Reach for Enhancing Healthcare Sustainability and Fairness](https://mlsanity.com/paper/2406.07558/) (2024)
- [Contactless Cardiac Pulse Monitoring Using Event Cameras](https://mlsanity.com/paper/2505.09529/) (2025)

## Abstract (arXiv)

Heartbeat rhythm and heart rate (HR) are important physiological parameters of the human body. This study presents an efficient multi-hierarchical spatio-temporal convolutional network that can quickly estimate remote physiological (rPPG) signal and HR from face video clips. First, the facial color distribution characteristics are extracted using a low-level face feature generation (LFFG) module. Then, the three-dimensional (3D) spatio-temporal stack convolution module (STSC) and multi-hierarchical feature fusion module (MHFF) are used to strengthen the spatio-temporal correlation of multi-channel features. In the MHFF, sparse optical flow is used to capture the tiny motion information of faces between frames and generate a self-adaptive region of interest (ROI) skin mask. Finally, the signal prediction module (SP) is used to extract the estimated rPPG signal. The heart rate estimation results show that the proposed network overperforms the state-of-the-art methods on three datasets, 1) UBFC-RPPG, 2) COHFACE, 3) our dataset, with the mean absolute error (MAE) of 2.15, 5.57, 1.75 beats per minute (bpm) respectively.