# DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks

arXiv: https://arxiv.org/abs/1805.07888 · HTML: https://mlsanity.com/paper/1805.07888/

## Key facts

- Cited by 154 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2018.

## Built on by (most cited first, 30 of 154)

- [PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer](https://mlsanity.com/paper/2111.12082/) (2021)
- [EfficientPhys: Enabling Simple, Fast and Accurate Camera-Based Vitals Measurement](https://mlsanity.com/paper/2110.04447/) (2021)
- [The Way to my Heart is through Contrastive Learning: Remote Photoplethysmography from Unlabelled Video](https://mlsanity.com/paper/2111.09748/) (2021)
- [rPPG-Toolbox: Deep Remote PPG Toolbox](https://mlsanity.com/paper/2210.00716/) (2022)
- [Camera Measurement of Physiological Vital Signs](https://mlsanity.com/paper/2111.11547/) (2021)
- [Contrast-Phys: Unsupervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast](https://mlsanity.com/paper/2208.04378/) (2022)
- [PhysFormer++: Facial Video-based Physiological Measurement with SlowFast Temporal Difference Transformer](https://mlsanity.com/paper/2302.03548/) (2023)
- [TransRPPG: Remote Photoplethysmography Transformer for 3D Mask Face Presentation Attack Detection](https://mlsanity.com/paper/2104.07419/) (2021)
- [Non-Contrastive Unsupervised Learning of Physiological Signals from Video](https://mlsanity.com/paper/2303.07944/) (2023)
- [rPPG-MAE: Self-supervised Pre-training with Masked Autoencoders for Remote Physiological Measurement](https://mlsanity.com/paper/2306.02301/) (2023)
- [Contrast-Phys+: Unsupervised and Weakly-supervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast](https://mlsanity.com/paper/2309.06924/) (2023)
- [Neuron Structure Modeling for Generalizable Remote Physiological Measurement](https://mlsanity.com/paper/2303.05955/) (2023)
- [RhythmFormer: Extracting Patterned rPPG Signals based on Periodic Sparse Attention](https://mlsanity.com/paper/2402.12788/) (2024)
- [SCAMPS: Synthetics for Camera Measurement of Physiological Signals](https://mlsanity.com/paper/2206.04197/) (2022)
- [Learning Motion-Robust Remote Photoplethysmography through Arbitrary Resolution Videos](https://mlsanity.com/paper/2211.16922/) (2022)
- [Self-supervised Representation Learning Framework for Remote Physiological Measurement Using Spatiotemporal Augmentation Loss](https://mlsanity.com/paper/2107.07695/) (2021)
- [Face2PPG: An unsupervised pipeline for blood volume pulse extraction from faces](https://mlsanity.com/paper/2202.04101/) (2022)
- [GPT as Psychologist? Preliminary Evaluations for GPT-4V on Visual Affective Computing](https://mlsanity.com/paper/2403.05916/) (2024)
- [RhythmMamba: Fast, Lightweight, and Accurate Remote Physiological Measurement](https://mlsanity.com/paper/2404.06483/) (2024)
- [The First Vision For Vitals (V4V) Challenge for Non-Contact Video-Based Physiological Estimation](https://mlsanity.com/paper/2109.10471/) (2021)
- [Spiking-PhysFormer: Camera-Based Remote Photoplethysmography with Parallel Spike-driven Transformer](https://mlsanity.com/paper/2402.04798/) (2024)
- [Dual-path TokenLearner for Remote Photoplethysmography-based Physiological Measurement with Facial Videos](https://mlsanity.com/paper/2308.07771/) (2023)
- [PhysMamba: Efficient Remote Physiological Measurement with SlowFast Temporal Difference Mamba](https://mlsanity.com/paper/2409.12031/) (2024)
- [FactorizePhys: Matrix Factorization for Multidimensional Attention in Remote Physiological Sensing](https://mlsanity.com/paper/2411.01542/) (2024)
- [BigSmall: Efficient Multi-Task Learning for Disparate Spatial and Temporal Physiological Measurements](https://mlsanity.com/paper/2303.11573/) (2023)
- [Advancing Generalizable Remote Physiological Measurement through the Integration of Explicit and Implicit Prior Knowledge](https://mlsanity.com/paper/2403.06947/) (2024)
- [Efficient Remote Photoplethysmography with Temporal Derivative Modules and Time-Shift Invariant Loss](https://mlsanity.com/paper/2203.10882/) (2022)
- [Motion Matters: Neural Motion Transfer for Better Camera Physiological Measurement](https://mlsanity.com/paper/2303.12059/) (2023)
- [egoPPG: Heart Rate Estimation from Eye-Tracking Cameras in Egocentric Systems to Benefit Downstream Vision Tasks](https://mlsanity.com/paper/2502.20879/) (2025)
- [Depression Recognition using Remote Photoplethysmography from Facial Videos](https://mlsanity.com/paper/2206.04399/) (2022)

## Abstract (arXiv)

Non-contact video-based physiological measurement has many applications in health care and human-computer interaction. Practical applications require measurements to be accurate even in the presence of large head rotations. We propose the first end-to-end system for video-based measurement of heart and breathing rate using a deep convolutional network. The system features a new motion representation based on a skin reflection model and a new attention mechanism using appearance information to guide motion estimation, both of which enable robust measurement under heterogeneous lighting and major motions. Our approach significantly outperforms all current state-of-the-art methods on both RGB and infrared video datasets. Furthermore, it allows spatial-temporal distributions of physiological signals to be visualized via the attention mechanism.