# PhysLLM: Harnessing Large Language Models for Cross-Modal Remote Physiological Sensing

arXiv: https://arxiv.org/abs/2505.03621 · HTML: https://mlsanity.com/paper/2505.03621/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 May 2025.

## Built on by (most cited first, 14 of 14)

- [PHASE-Net: Physics-Grounded Harmonic Attention System for Efficient Remote Photoplethysmography Measurement](https://mlsanity.com/paper/2509.24850/) (2025)
- [AU-LLM: Micro-Expression Action Unit Detection via Enhanced LLM-Based Feature Fusion](https://mlsanity.com/paper/2507.21778/) (2025)
- [FreqPhys: Repurposing Implicit Physiological Frequency Prior for Robust Remote Photoplethysmography](https://mlsanity.com/paper/2604.00534/) (2026)
- [HOT: Harmonic-Constrained Optimal Transport for Remote Photoplethysmography Domain Adaptation](https://mlsanity.com/paper/2604.01675/) (2026)
- [PhysFlow: Frequency Decoupled with Dual-Field Rectified Flow for Remote Photoplethysmography](https://mlsanity.com/paper/2606.23226/) (2026)
- [BTS-rPPG: Orthogonal Butterfly Temporal Shifting for Remote Photoplethysmography](https://mlsanity.com/paper/2604.01679/) (2026)
- [SVC 2026: the Second Multimodal Deception Detection Challenge and the First Domain Generalized Remote Physiological Measurement Challenge](https://mlsanity.com/paper/2604.05748/) (2026)
- [PCA: Persistence-Aware Compression and Aggregation for Fast Video Large Language Models](https://mlsanity.com/paper/2607.22726/) (2026)
- [CardiacMamba: Fair and Robust RGB-RF Fusion for Remote Heart Rate Estimation via State Space Modeling](https://mlsanity.com/paper/2608.15831/) (2026)
- [PhysVR: Vision-Language Model Guided Interference-aware Temporal Feature Refinement for Remote Physiological Measurement](https://mlsanity.com/paper/2608.29663/) (2026)
- [Large language models in medical time series analysis](https://mlsanity.com/paper/2609.22262/) (2026)
- [PPG-LM: A Photoplethysmography-Language Model with Multi-Level Clinical Alignment](https://mlsanity.com/paper/2609.33516/) (2026)
- [FLOW: Feature-Level Optimal Warping for Generalized Remote Physiological Measurement](https://mlsanity.com/paper/2609.38913/) (2026)
- [Why Do VLMs Fail at Remote Physiological Measurement? Rethinking with Physiological Grounding and Numerical Alignment](https://mlsanity.com/paper/2610.19052/) (2026)

## Abstract (arXiv)

Remote photoplethysmography (rPPG) enables non-contact physiological measurement but remains highly susceptible to illumination changes, motion artifacts, and limited temporal modeling. Large Language Models (LLMs) excel at capturing long-range dependencies, offering a potential solution but struggle with the continuous, noise-sensitive nature of rPPG signals due to their text-centric design. To bridge this gap, we introduce the PhysLLM, a collaborative optimization framework that synergizes LLMs with domain-specific rPPG components. Specifically, the Text Prototype Guidance (TPG) strategy is proposed to establish cross-modal alignment by projecting hemodynamic features into LLM-interpretable semantic space, effectively bridging the representational gap between physiological signals and linguistic tokens. Besides, a novel Dual-Domain Stationary (DDS) Algorithm is proposed for resolving signal instability through adaptive time-frequency domain feature re-weighting. Finally, rPPG task-specific cues systematically inject physiological priors through physiological statistics, environmental contextual answering, and task description, leveraging cross-modal learning to integrate both visual and textual information, enabling dynamic adaptation to challenging scenarios like variable illumination and subject movements. Evaluation on four benchmark datasets, PhysLLM achieves state-of-the-art accuracy and robustness, demonstrating superior generalization across lighting variations and motion scenarios. The source code is available at https://github.com/Alex036225/PhysLLM.