# KDPhys: An Attention Guided 3D to 2D Knowledge Distillation for Real-time Video-Based Physiological Measurement arXiv: https://arxiv.org/abs/2601.00714 ยท HTML: https://mlsanity.com/paper/2601.00714/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation. - Builds on 31 of its 71 references that are in the map. - First posted on arXiv 2 Jan 2026. ## Placed in - Knowledge distillation / [Distillation for medical imaging](https://mlsanity.com/distill/distillation-for-medical-imaging/) ## Builds on (most cited first, 30 of 31) - [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015) - [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017) - [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014) - [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016) - [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019) - [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018) - [Contrastive Representation Distillation](https://mlsanity.com/paper/1910.10699/) (2019) - [DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks](https://mlsanity.com/paper/1805.07888/) (2018) - [Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement](https://mlsanity.com/paper/2006.03790/) (2020) - [Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks](https://mlsanity.com/paper/1905.02419/) (2019) - [PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer](https://mlsanity.com/paper/2111.12082/) (2021) - [Remote Heart Rate Measurement from Highly Compressed Facial Videos: an End-to-end Deep Learning Solution with Video Enhancement](https://mlsanity.com/paper/1907.11921/) (2019) - [Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling](https://mlsanity.com/paper/2007.08213/) (2020) - [Show, Attend and Distill:Knowledge Distillation via Attention-based Feature Matching](https://mlsanity.com/paper/2102.02973/) (2021) - [EfficientPhys: Enabling Simple, Fast and Accurate Camera-Based Vitals Measurement](https://mlsanity.com/paper/2110.04447/) (2021) - [Shape and Time Distortion Loss for Training Deep Time Series Forecasting Models](https://mlsanity.com/paper/1909.09020/) (2019) - [ALP-KD: Attention-Based Layer Projection for Knowledge Distillation](https://mlsanity.com/paper/2012.14022/) (2020) - [PulseGAN: Learning to generate realistic pulse waveforms in remote photoplethysmography](https://mlsanity.com/paper/2006.02699/) (2020) - [MMPD: Multi-Domain Mobile Video Physiology Dataset](https://mlsanity.com/paper/2302.03840/) (2023) - [A Reproducible Study on Remote Heart Rate Measurement](https://mlsanity.com/paper/1709.00962/) (2017) - [3D-to-2D Distillation for Indoor Scene Parsing](https://mlsanity.com/paper/2104.02243/) (2021) - [Contrast-Phys: Unsupervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast](https://mlsanity.com/paper/2208.04378/) (2022) - [PhysFormer++: Facial Video-based Physiological Measurement with SlowFast Temporal Difference Transformer](https://mlsanity.com/paper/2302.03548/) (2023) - [Cross-Architecture Knowledge Distillation](https://mlsanity.com/paper/2207.05273/) (2022) - [Neuron Structure Modeling for Generalizable Remote Physiological Measurement](https://mlsanity.com/paper/2303.05955/) (2023) - [Self-supervised Representation Learning Framework for Remote Physiological Measurement Using Spatiotemporal Augmentation Loss](https://mlsanity.com/paper/2107.07695/) (2021) - [Frequency Attention for Knowledge Distillation](https://mlsanity.com/paper/2403.05894/) (2024) - [Face2PPG: An unsupervised pipeline for blood volume pulse extraction from faces](https://mlsanity.com/paper/2202.04101/) (2022) - [KD-MRI: A knowledge distillation framework for image reconstruction and image restoration in MRI workflow](https://mlsanity.com/paper/2004.05319/) (2020) - [iPhys: An Open Non-Contact Imaging-Based Physiological Measurement Toolbox](https://mlsanity.com/paper/1901.04366/) (2019) ## Built on by (most cited first, 1 of 1) - [Cross-Dataset Transfer and Reliability of Explainable Artificial Intelligence for RhythmFormer Remote Photoplethysmography](https://mlsanity.com/paper/2609.03663/) (2026) ## Abstract (arXiv) Camera-based physiological monitoring, such as remote photoplethysmography (rPPG), captures subtle variations in skin optical properties caused by pulsatile blood volume changes using standard digital camera sensors. The demand for real-time, non-contact physiological measurement has increased significantly, particularly during the SARS-CoV-2 pandemic, to support telehealth and remote health monitoring applications. In this work, we propose an attention-based knowledge distillation (KD) framework, termed KDPhys, for extracting rPPG signals from facial video sequences. The proposed method distills global temporal representations from a 3D convolutional neural network (CNN) teacher model to a lightweight 2D CNN student model through effective 3D-to-2D feature distillation. To the best of our knowledge, this is the first application of knowledge distillation in the rPPG domain. Furthermore, we introduce a Distortion Loss incorporating Shape and Time (DILATE), which jointly accounts for both morphological and temporal characteristics of rPPG signals. Extensive qualitative and quantitative evaluations are conducted on three benchmark datasets. The proposed model achieves a significant reduction in computational complexity, using only half the parameters of existing methods while operating 56.67% faster. With just 0.23M parameters, it achieves an 18.15% reduction in Mean Absolute Error (MAE) compared to state-of-the-art approaches, attaining an average MAE of 1.78 bpm across all datasets. Additional experiments under diverse environmental conditions and activity scenarios further demonstrate the robustness and adaptability of the proposed approach.