# PPG-Distill: Efficient Photoplethysmography Signals Analysis via Foundation Model Distillation

arXiv: https://arxiv.org/abs/2509.19215 · HTML: https://mlsanity.com/paper/2509.19215/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation.
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 23 Sep 2025.

## Placed in

- Knowledge distillation / [Distillation for medical imaging](https://mlsanity.com/distill/distillation-for-medical-imaging/)

## Builds on (most cited first, 18 of 18)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [Large-scale Training of Foundation Models for Wearable Biosignals](https://mlsanity.com/paper/2312.05409/) (2023)
- [PaPaGei: Open Foundation Models for Optical Physiological Signals](https://mlsanity.com/paper/2410.20542/) (2024)
- [LightTS: Lightweight Time Series Classification with Adaptive Ensemble Distillation -- Extended Version](https://mlsanity.com/paper/2302.12721/) (2023)
- [Pulse-PPG: An Open-Source Field-Trained PPG Foundation Model for Wearable Applications Across Lab and Field Settings](https://mlsanity.com/paper/2502.01108/) (2025)
- [Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions](https://mlsanity.com/paper/2507.00191/) (2025)
- [Graph ODEs and Beyond: A Comprehensive Survey on Integrating Differential Equations with Graph Neural Networks](https://mlsanity.com/paper/2503.23167/) (2025)
- [TimeDistill: Efficient Long-Term Time Series Forecasting with MLP via Cross-Architecture Distillation](https://mlsanity.com/paper/2502.15016/) (2025)
- [Can Large Language Models Adequately Perform Symbolic Reasoning Over Time Series?](https://mlsanity.com/paper/2508.03963/) (2025)
- [U-Cast: Learning Hierarchical Structures for High-Dimensional Time Series Forecasting](https://mlsanity.com/paper/2507.15119/) (2025)
- [Pre-training Epidemic Time Series Forecasters with Compartmental Prototypes](https://mlsanity.com/paper/2502.03393/) (2025)
- [SeizureFormer: A Transformer Model for IEA-Based Seizure Risk Forecasting](https://mlsanity.com/paper/2504.16098/) (2025)

## Built on by (most cited first, 6 of 6)

- [Simulator and Experience Enhanced Diffusion Model for Comprehensive ECG Generation](https://mlsanity.com/paper/2511.09895/) (2025)
- [Generalist vs Specialist Time Series Foundation Models: Investigating Potential Emergent Behaviors in Assessing Human Health Using PPG Signals](https://mlsanity.com/paper/2510.14254/) (2025)
- [Wav2Arrest 2.0: Long-Horizon Cardiac Arrest Prediction with Time-to-Event Modeling, Identity-Invariance, and Pseudo-Lab Alignment](https://mlsanity.com/paper/2509.21695/) (2025)
- [ECG-MoE: Mixture-of-Expert Electrocardiogram Foundation Model](https://mlsanity.com/paper/2603.04589/) (2026)
- [Multi-site PPG: An In-the-Wild Physiological Dataset from Emerging Multi-site Wearables](https://mlsanity.com/paper/2605.17859/) (2026)
- [When Derived Measurements Mislead: Quantifying and Mitigating LLM Over-Trust with Privileged-Modality Reliability Evidence](https://mlsanity.com/paper/2607.28421/) (2026)

## Abstract (arXiv)

Photoplethysmography (PPG) is widely used in wearable health monitoring, yet large PPG foundation models remain difficult to deploy on resource-limited devices. We present PPG-Distill, a knowledge distillation framework that transfers both global and local knowledge through prediction-, feature-, and patch-level distillation. PPG-Distill incorporates morphology distillation to preserve local waveform patterns and rhythm distillation to capture inter-patch temporal structures. On heart rate estimation and atrial fibrillation detection, PPG-Distill improves student performance by up to 21.8% while achieving 7X faster inference and reducing memory usage by 19X, enabling efficient PPG analysis on wearables.