# Motion Matters: Neural Motion Transfer for Better Camera Physiological Measurement

arXiv: https://arxiv.org/abs/2303.12059 · HTML: https://mlsanity.com/paper/2303.12059/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 60 references that are in the map.
- First posted on arXiv 21 Mar 2023.

## Builds on (most cited first, 30 of 35)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing](https://mlsanity.com/paper/2011.15126/) (2020)
- [Few-Shot Adversarial Learning of Realistic Neural Talking Head Models](https://mlsanity.com/paper/1905.08233/) (2019)
- [Fake It Till You Make It: Face analysis in the wild using synthetic data alone](https://mlsanity.com/paper/2109.15102/) (2021)
- [Depth-Aware Generative Adversarial Network for Talking Head Video Generation](https://mlsanity.com/paper/2203.06605/) (2022)
- [DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks](https://mlsanity.com/paper/1805.07888/) (2018)
- [FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals](https://mlsanity.com/paper/1901.02212/) (2019)
- [Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement](https://mlsanity.com/paper/2006.03790/) (2020)
- [PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer](https://mlsanity.com/paper/2111.12082/) (2021)
- [Remote Heart Rate Measurement from Highly Compressed Facial Videos: an End-to-end Deep Learning Solution with Video Enhancement](https://mlsanity.com/paper/1907.11921/) (2019)
- [EfficientPhys: Enabling Simple, Fast and Accurate Camera-Based Vitals Measurement](https://mlsanity.com/paper/2110.04447/) (2021)
- [The Way to my Heart is through Contrastive Learning: Remote Photoplethysmography from Unlabelled Video](https://mlsanity.com/paper/2111.09748/) (2021)
- [rPPG-Toolbox: Deep Remote PPG Toolbox](https://mlsanity.com/paper/2210.00716/) (2022)
- [VIPL-HR: A Multi-modal Database for Pulse Estimation from Less-constrained Face Video](https://mlsanity.com/paper/1810.04927/) (2018)
- [MMPD: Multi-Domain Mobile Video Physiology Dataset](https://mlsanity.com/paper/2302.03840/) (2023)
- [Rendering of Eyes for Eye-Shape Registration and Gaze Estimation](https://mlsanity.com/paper/1505.05916/) (2015)
- [Camera Measurement of Physiological Vital Signs](https://mlsanity.com/paper/2111.11547/) (2021)
- [Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications](https://mlsanity.com/paper/2008.02793/) (2020)
- [Behavioral Use Licensing for Responsible AI](https://mlsanity.com/paper/2011.03116/) (2020)
- [SimPer: Simple Self-Supervised Learning of Periodic Targets](https://mlsanity.com/paper/2210.03115/) (2022)
- [Non-Contrastive Unsupervised Learning of Physiological Signals from Video](https://mlsanity.com/paper/2303.07944/) (2023)
- [Studying Bias in GANs through the Lens of Race](https://mlsanity.com/paper/2209.02836/) (2022)
- [SCAMPS: Synthetics for Camera Measurement of Physiological Signals](https://mlsanity.com/paper/2206.04197/) (2022)

## Built on by (most cited first, 11 of 11)

- [My3DGen: A Scalable Personalized 3D Generative Model](https://mlsanity.com/paper/2307.05468/) (2023)
- [Orientation-conditioned Facial Texture Mapping for Video-based Facial Remote Photoplethysmography Estimation](https://mlsanity.com/paper/2404.09378/) (2024)
- [Toward Motion Robustness: A masked attention regularization framework in remote photoplethysmography](https://mlsanity.com/paper/2407.06653/) (2024)
- [PhysFlow: Skin tone transfer for remote heart rate estimation through conditional normalizing flows](https://mlsanity.com/paper/2407.21519/) (2024)
- [BeatFormer: Efficient motion-robust remote heart rate estimation through unsupervised spectral zoomed attention filters](https://mlsanity.com/paper/2507.14885/) (2025)
- [Editing Physiological Signals in Videos Using Latent Representations](https://mlsanity.com/paper/2509.25348/) (2025)
- [Thermal Imaging and Radar for Remote Sleep Monitoring of Breathing and Apnea](https://mlsanity.com/paper/2407.11936/) (2024)
- [CAST-Phys: Contactless Affective States Through Physiological signals Database](https://mlsanity.com/paper/2507.06080/) (2025)
- [Facial Spatiotemporal Graphs: Leveraging the 3D Facial Surface for Remote Physiological Measurement](https://mlsanity.com/paper/2601.13724/) (2026)
- [Beyond Ambiguous Visual Cues: Studying Physiological Disruptions and Cross-Modal Inconsistencies in Deepfake Videos](https://mlsanity.com/paper/2609.12668/) (2026)
- [Heartian: Physiology-Aware Relightable Gaussian Head Avatar](https://mlsanity.com/paper/2609.28539/) (2026)

## Abstract (arXiv)

Machine learning models for camera-based physiological measurement can have weak generalization due to a lack of representative training data. Body motion is one of the most significant sources of noise when attempting to recover the subtle cardiac pulse from a video. We explore motion transfer as a form of data augmentation to introduce motion variation while preserving physiological changes of interest. We adapt a neural video synthesis approach to augment videos for the task of remote photoplethysmography (rPPG) and study the effects of motion augmentation with respect to 1) the magnitude and 2) the type of motion. After training on motion-augmented versions of publicly available datasets, we demonstrate a 47% improvement over existing inter-dataset results using various state-of-the-art methods on the PURE dataset. We also present inter-dataset results on five benchmark datasets to show improvements of up to 79% using TS-CAN, a neural rPPG estimation method. Our findings illustrate the usefulness of motion transfer as a data augmentation technique for improving the generalization of models for camera-based physiological sensing. We release our code for using motion transfer as a data augmentation technique on three publicly available datasets, UBFC-rPPG, PURE, and SCAMPS, and models pre-trained on motion-augmented data here: https://motion-matters.github.io/