# Dual-path TokenLearner for Remote Photoplethysmography-based Physiological Measurement with Facial Videos

arXiv: https://arxiv.org/abs/2308.07771 · HTML: https://mlsanity.com/paper/2308.07771/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 79 references that are in the map.
- First posted on arXiv 15 Aug 2023.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks](https://mlsanity.com/paper/1805.07888/) (2018)
- [Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement](https://mlsanity.com/paper/2006.03790/) (2020)
- [Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks](https://mlsanity.com/paper/1905.02419/) (2019)
- [PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer](https://mlsanity.com/paper/2111.12082/) (2021)
- [RhythmNet: End-to-end Heart Rate Estimation from Face via Spatial-temporal Representation](https://mlsanity.com/paper/1910.11515/) (2019)
- [Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling](https://mlsanity.com/paper/2007.08213/) (2020)
- [Domain Generalization via Shuffled Style Assembly for Face Anti-Spoofing](https://mlsanity.com/paper/2203.05340/) (2022)
- [The Way to my Heart is through Contrastive Learning: Remote Photoplethysmography from Unlabelled Video](https://mlsanity.com/paper/2111.09748/) (2021)
- [Deep Learning for Face Anti-Spoofing: A Survey](https://mlsanity.com/paper/2106.14948/) (2021)
- [PulseGAN: Learning to generate realistic pulse waveforms in remote photoplethysmography](https://mlsanity.com/paper/2006.02699/) (2020)
- [VIPL-HR: A Multi-modal Database for Pulse Estimation from Less-constrained Face Video](https://mlsanity.com/paper/1810.04927/) (2018)
- [Meta-rPPG: Remote Heart Rate Estimation Using a Transductive Meta-Learner](https://mlsanity.com/paper/2007.06786/) (2020)
- [Contrast-Phys: Unsupervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast](https://mlsanity.com/paper/2208.04378/) (2022)
- [TransRPPG: Remote Photoplethysmography Transformer for 3D Mask Face Presentation Attack Detection](https://mlsanity.com/paper/2104.07419/) (2021)
- [AutoHR: A Strong End-to-end Baseline for Remote Heart Rate Measurement with Neural Searching](https://mlsanity.com/paper/2004.12292/) (2020)
- [Neuron Structure Modeling for Generalizable Remote Physiological Measurement](https://mlsanity.com/paper/2303.05955/) (2023)
- [Learning Motion-Robust Remote Photoplethysmography through Arbitrary Resolution Videos](https://mlsanity.com/paper/2211.16922/) (2022)
- [ViGT: Proposal-free Video Grounding with Learnable Token in Transformer](https://mlsanity.com/paper/2308.06009/) (2023)
- [Advancing Non-Contact Vital Sign Measurement using Synthetic Avatars](https://mlsanity.com/paper/2010.12949/) (2020)
- [RGB Camera-based Physiological Sensing: Challenges and Future Directions](https://mlsanity.com/paper/2110.13362/) (2021)

## Built on by (most cited first, 15 of 15)

- [EulerMormer: Robust Eulerian Motion Magnification via Dynamic Filtering within Transformer](https://mlsanity.com/paper/2312.04152/) (2023)
- [Orientation-conditioned Facial Texture Mapping for Video-based Facial Remote Photoplethysmography Estimation](https://mlsanity.com/paper/2404.09378/) (2024)
- [FreqPhys: Repurposing Implicit Physiological Frequency Prior for Robust Remote Photoplethysmography](https://mlsanity.com/paper/2604.00534/) (2026)
- [Phase-shifted remote photoplethysmography for estimating heart rate and blood pressure from facial video](https://mlsanity.com/paper/2401.04560/) (2024)
- [Text-guided Fine-Grained Video Anomaly Understanding](https://mlsanity.com/paper/2511.00524/) (2025)
- [Remote Heart Rate Monitoring in Smart Environments from Videos with Self-supervised Pre-training](https://mlsanity.com/paper/2310.15388/) (2023)
- [Joint Spatial-Temporal Modeling and Contrastive Learning for Self-supervised Heart Rate Measurement](https://mlsanity.com/paper/2406.04942/) (2024)
- [HOT: Harmonic-Constrained Optimal Transport for Remote Photoplethysmography Domain Adaptation](https://mlsanity.com/paper/2604.01675/) (2026)
- [PhysFlow: Frequency Decoupled with Dual-Field Rectified Flow for Remote Photoplethysmography](https://mlsanity.com/paper/2606.23226/) (2026)
- [Exploiting Diverse Feature for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2308.13421/) (2023)
- [Bootstrapping Vision-language Models for Self-supervised Remote Physiological Measurement](https://mlsanity.com/paper/2407.08507/) (2024)
- [Facial Spatiotemporal Graphs: Leveraging the 3D Facial Surface for Remote Physiological Measurement](https://mlsanity.com/paper/2601.13724/) (2026)
- [BTS-rPPG: Orthogonal Butterfly Temporal Shifting for Remote Photoplethysmography](https://mlsanity.com/paper/2604.01679/) (2026)
- [SVC 2026: the Second Multimodal Deception Detection Challenge and the First Domain Generalized Remote Physiological Measurement Challenge](https://mlsanity.com/paper/2604.05748/) (2026)
- [Explaining RhythmFormer: A Systematic XAI Analysis of Periodic Sparse Attention for Remote Photoplethysmography](https://mlsanity.com/paper/2606.13839/) (2026)

## Abstract (arXiv)

Remote photoplethysmography (rPPG) based physiological measurement is an emerging yet crucial vision task, whose challenge lies in exploring accurate rPPG prediction from facial videos accompanied by noises of illumination variations, facial occlusions, head movements, etc, in a non-contact manner. Existing mainstream CNN-based models make efforts to detect physiological signals by capturing subtle color changes in facial regions of interest (ROI) caused by heartbeats. However, such models are constrained by the limited local spatial or temporal receptive fields in the neural units. Unlike them, a native Transformer-based framework called Dual-path TokenLearner (Dual-TL) is proposed in this paper, which utilizes the concept of learnable tokens to integrate both spatial and temporal informative contexts from the global perspective of the video. Specifically, the proposed Dual-TL uses a Spatial TokenLearner (S-TL) to explore associations in different facial ROIs, which promises the rPPG prediction far away from noisy ROI disturbances. Complementarily, a Temporal TokenLearner (T-TL) is designed to infer the quasi-periodic pattern of heartbeats, which eliminates temporal disturbances such as head movements. The two TokenLearners, S-TL and T-TL, are executed in a dual-path mode. This enables the model to reduce noise disturbances for final rPPG signal prediction. Extensive experiments on four physiological measurement benchmark datasets are conducted. The Dual-TL achieves state-of-the-art performances in both intra- and cross-dataset testings, demonstrating its immense potential as a basic backbone for rPPG measurement. The source code is available at hrefhttps://github.com/VUT-HFUT/Dual-TLhttps://github.com/VUT-HFUT/Dual-TL