# Your smartphone could act as a pulse-oximeter and as a single-lead ECG

arXiv: https://arxiv.org/abs/2305.12583 · HTML: https://mlsanity.com/paper/2305.12583/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 55 references that are in the map.
- First posted on arXiv 21 May 2023.

## Builds on (most cited first, 7 of 7)

- [AudioCLIP: Extending CLIP to Image, Text and Audio](https://mlsanity.com/paper/2106.13043/) (2021)
- [Evaluation of PPG Biometrics for Authentication in different states](https://mlsanity.com/paper/1712.08583/) (2017)
- [TransPPG: Two-stream Transformer for Remote Heart Rate Estimate](https://mlsanity.com/paper/2201.10873/) (2022)
- [A Deep Learning & Fast Wavelet Transform-based Hybrid Approach for Denoising of PPG Signals](https://mlsanity.com/paper/2301.06549/) (2023)
- [ECG Reconstruction via PPG: A Pilot Study](https://mlsanity.com/paper/1904.10481/) (2019)
- [A Comparative Evaluation of Heart Rate Estimation Methods using Face Videos](https://mlsanity.com/paper/2005.11101/) (2020)
- [Efficient Deep Learning-based Estimation of the Vital Signs on Smartphones](https://mlsanity.com/paper/2204.08989/) (2022)

## Built on by (most cited first, 3 of 3)

- [TCM-Ladder: A Benchmark for Multimodal Question Answering on Traditional Chinese Medicine](https://mlsanity.com/paper/2505.24063/) (2025)
- [Cuff-less Arterial Blood Pressure Waveform Synthesis from Single-site PPG using Transformer & Frequency-domain Learning](https://mlsanity.com/paper/2401.05452/) (2024)
- [You can monitor your hydration level using your smartphone camera](https://mlsanity.com/paper/2402.07467/) (2024)

## Abstract (arXiv)

In the post-covid19 era, every new wave of the pandemic causes an increased concern among the masses to learn more about their state of well-being. Therefore, it is the need of the hour to come up with ubiquitous, low-cost, non-invasive tools for rapid and continuous monitoring of body vitals that reflect the status of one's overall health. In this backdrop, this work proposes a deep learning approach to turn a smartphone-the popular hand-held personal gadget-into a diagnostic tool to measure/monitor the three most important body vitals, i.e., pulse rate (PR), blood oxygen saturation level (aka SpO2), and respiratory rate (RR). Furthermore, we propose another method that could extract a single-lead electrocardiograph (ECG) of the subject. The proposed methods include the following core steps: subject records a small video of his/her fingertip by placing his/her finger on the rear camera of the smartphone, and the recorded video is pre-processed to extract the filtered and/or detrended video-photoplethysmography (vPPG) signal, which is then fed to custom-built convolutional neural networks (CNN), which eventually spit-out the vitals (PR, SpO2, and RR) as well as a single-lead ECG of the subject. To be precise, the contribution of this paper is two-fold: 1) estimation of the three body vitals (PR, SpO2, RR) from the vPPG data using custom-built CNNs, vision transformer, and most importantly by CLIP model; 2) a novel discrete cosine transform+feedforward neural network-based method that translates the recorded video- PPG signal to a single-lead ECG signal. The proposed method is anticipated to find its application in several use-case scenarios, e.g., remote healthcare, mobile health, fitness, sports, etc.