mlsanityData to 14 Oct 2026

Towards Characterizing Knowledge Distillation of PPG Heart Rate Estimation Models

arXiv 2511.18829 ↗First posted 24 Nov 2025cs.LGcited by 0

On the map: Knowledge distillation → Distillation for medical imaging

Abstract (arXiv)

Heart rate estimation from photoplethysmography (PPG) signals generated by wearable devices such as smartwatches and fitness trackers has significant implications for the health and well-being of individuals. Although prior work has demonstrated deep learning models with strong performance in the heart rate estimation task, in order to deploy these models on wearable devices, these models must also adhere to strict memory and latency constraints. In this work, we explore and characterize how large pre-trained PPG models may be distilled to smaller models appropriate for real-time inference on the edge. We evaluate four distillation strategies through comprehensive sweeps of teacher and student model capacities: (1) hard distillation, (2) soft distillation, (3) decoupled knowledge distillation (DKD), and (4) feature distillation. We present a characterization of the resulting scaling laws describing the relationship between model size and performance. This early investigation lays the groundwork for practical and predictable methods for building edge-deployable models for physiological sensing.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation. Builds on 13 of its 19 references that are in the map. First posted on arXiv 24 Nov 2025.

Builds on

13 of 19 references.

References · 13

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  4. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  5. 2020Knowledge Distillation: A Surveycited by 1,603
  6. 2024PaPaGei: Open Foundation Models for Optical Physiological Signalscited by 69
  7. 2025Distillation Scaling Lawscited by 67
  8. 2024Scaling Wearable Foundation Modelscited by 63
  9. 2025SensorLM: Learning the Language of Wearable Sensorscited by 49
  10. 2025Pulse-PPG: An Open-Source Field-Trained PPG Foundation Model for Wearable Applications Across Lab and Field Settingscited by 37
  11. 2025LSM-2: Learning from Incomplete Wearable Sensor Datacited by 23
  12. 2024WildPPG: A Real-World PPG Dataset of Long Continuous Recordingscited by 15
1 more
  1. 2020FRILL: A Non-Semantic Speech Embedding for Mobile Devicescited by 13

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025PPG-Distill: Efficient Photoplethysmography Signals Analysis via Foundation Model Distillation
  2. 2022Q-PPG: Energy-Efficient PPG-based Heart Rate Monitoring on Wearable Devices
  3. 2024Wearable Accelerometer Foundation Models for Health via Knowledge Distillation
  4. 2025Robust Photoplethysmography Signal Denoising via Mamba Networks
  5. 2022Role of Data Augmentation Strategies in Knowledge Distillation for Wearable Sensor Data
  6. 2023BeliefPPG: Uncertainty-aware Heart Rate Estimation from PPG signals via Belief Propagation
  7. 2026Towards a General Intelligence and Interface for Wearable Health Data
  8. 2024Optimization and Deployment of Deep Neural Networks for PPG-based Blood Pressure Estimation Targeting Low-power Wearables