# Synthetic Data for Multi-Parameter Camera-Based Physiological Sensing

arXiv: https://arxiv.org/abs/2110.04902 · HTML: https://mlsanity.com/paper/2110.04902/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 42 references that are in the map.
- First posted on arXiv 10 Oct 2021.

## Builds on (most cited first, 16 of 16)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Learning from Simulated and Unsupervised Images through Adversarial Training](https://mlsanity.com/paper/1612.07828/) (2016)
- [DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks](https://mlsanity.com/paper/1805.07888/) (2018)
- [Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement](https://mlsanity.com/paper/2006.03790/) (2020)
- [UnrealCV: Connecting Computer Vision to Unreal Engine](https://mlsanity.com/paper/1609.01326/) (2016)
- [SceneNet: Understanding Real World Indoor Scenes With Synthetic Data](https://mlsanity.com/paper/1511.07041/) (2015)
- [Empirically Analyzing the Effect of Dataset Biases on Deep Face Recognition Systems](https://mlsanity.com/paper/1712.01619/) (2017)
- [Characterizing Bias in Classifiers using Generative Models](https://mlsanity.com/paper/1906.11891/) (2019)
- [Analysis of CNN-based remote-PPG to understand limitations and sensitivities](https://mlsanity.com/paper/1911.02736/) (2019)
- [Advancing Non-Contact Vital Sign Measurement using Synthetic Avatars](https://mlsanity.com/paper/2010.12949/) (2020)
- [Identifying Bias in AI using Simulation](https://mlsanity.com/paper/1810.00471/) (2018)
- [Simulations for Validation of Vision Systems](https://mlsanity.com/paper/1512.01030/) (2015)
- [Model Validation for Vision Systems via Graphics Simulation](https://mlsanity.com/paper/1512.01401/) (2015)
- [Model-driven Simulations for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1605.09582/) (2016)
- [A high fidelity synthetic face framework for computer vision](https://mlsanity.com/paper/2007.08364/) (2020)

## Built on by (most cited first, 9 of 9)

- [Camera Measurement of Physiological Vital Signs](https://mlsanity.com/paper/2111.11547/) (2021)
- [SCAMPS: Synthetics for Camera Measurement of Physiological Signals](https://mlsanity.com/paper/2206.04197/) (2022)
- [Synthetic Data in Human Analysis: A Survey](https://mlsanity.com/paper/2208.09191/) (2022)
- [AI-Generated Images as Data Source: The Dawn of Synthetic Era](https://mlsanity.com/paper/2310.01830/) (2023)
- [Motion Matters: Neural Motion Transfer for Better Camera Physiological Measurement](https://mlsanity.com/paper/2303.12059/) (2023)
- [Skin Deep: Investigating Subjectivity in Skin Tone Annotations for Computer Vision Benchmark Datasets](https://mlsanity.com/paper/2305.09072/) (2023)
- [Procedural Humans for Computer Vision](https://mlsanity.com/paper/2301.01161/) (2023)
- [RGB Camera-based Physiological Sensing: Challenges and Future Directions](https://mlsanity.com/paper/2110.13362/) (2021)
- [Training Robust Deep Physiological Measurement Models with Synthetic Video-based Data](https://mlsanity.com/paper/2311.05371/) (2023)

## Abstract (arXiv)

Synthetic data is a powerful tool in training data hungry deep learning algorithms. However, to date, camera-based physiological sensing has not taken full advantage of these techniques. In this work, we leverage a high-fidelity synthetics pipeline for generating videos of faces with faithful blood flow and breathing patterns. We present systematic experiments showing how physiologically-grounded synthetic data can be used in training camera-based multi-parameter cardiopulmonary sensing. We provide empirical evidence that heart and breathing rate measurement accuracy increases with the number of synthetic avatars in the training set. Furthermore, training with avatars with darker skin types leads to better overall performance than training with avatars with lighter skin types. Finally, we discuss the opportunities that synthetics present in the domain of camera-based physiological sensing and limitations that need to be overcome.