# CardioState-JEPA: Delay-Aware Cross-Modal Learning of a Shared Cardiac Representation

arXiv: https://arxiv.org/abs/2608.12944 · HTML: https://mlsanity.com/paper/2608.12944/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 53 references that are in the map.
- First posted on arXiv 13 Aug 2026.

## Builds on (most cited first, 30 of 30)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [Time-Series Representation Learning via Temporal and Contextual Contrasting](https://mlsanity.com/paper/2106.14112/) (2021)
- [Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts](https://mlsanity.com/paper/2409.16040/) (2024)
- [Masked Autoencoders that Listen](https://mlsanity.com/paper/2207.06405/) (2022)
- [Chronos-2: From Univariate to Universal Forecasting](https://mlsanity.com/paper/2510.15821/) (2025)
- [CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patients](https://mlsanity.com/paper/2005.13249/) (2020)
- [CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting](https://mlsanity.com/paper/2202.01575/) (2022)
- [ECG-FM: An Open Electrocardiogram Foundation Model](https://mlsanity.com/paper/2408.05178/) (2024)
- [Guiding Masked Representation Learning to Capture Spatio-Temporal Relationship of Electrocardiogram](https://mlsanity.com/paper/2402.09450/) (2024)
- [PaPaGei: Open Foundation Models for Optical Physiological Signals](https://mlsanity.com/paper/2410.20542/) (2024)
- [Zero-Shot ECG Classification with Multimodal Learning and Test-time Clinical Knowledge Enhancement](https://mlsanity.com/paper/2403.06659/) (2024)
- [Self-Supervised Time Series Representation Learning via Cross Reconstruction Transformer](https://mlsanity.com/paper/2205.09928/) (2022)
- [Pulse-PPG: An Open-Source Field-Trained PPG Foundation Model for Wearable Applications Across Lab and Field Settings](https://mlsanity.com/paper/2502.01108/) (2025)
- [Reading Your Heart: Learning ECG Words and Sentences via Pre-training ECG Language Model](https://mlsanity.com/paper/2502.10707/) (2025)
- [ECG Semantic Integrator (ESI): A Foundation ECG Model Pretrained with LLM-Enhanced Cardiological Text](https://mlsanity.com/paper/2405.19366/) (2024)
- [Boosting Masked ECG-Text Auto-Encoders as Discriminative Learners](https://mlsanity.com/paper/2410.02131/) (2024)
- [AnyPPG: An ECG-Guided PPG Foundation Model Trained on Over 100,000 Hours of Recordings for Holistic Health Profiling](https://mlsanity.com/paper/2511.01747/) (2025)
- [anyECG-chat: A Generalist ECG-MLLM for Flexible ECG Input and Multi-Task Understanding](https://mlsanity.com/paper/2506.00942/) (2025)
- [PulseLM: A Foundation Dataset and Benchmark for PPG-Text Learning](https://mlsanity.com/paper/2603.03331/) (2026)
- [StethoLM: Audio Language Model for Cardiopulmonary Analysis Across Clinical Tasks](https://mlsanity.com/paper/2603.00355/) (2026)

## Built on by (most cited first, 3 of 3)

- [Capturing Cardiac Cyclicity through Phase-Equivariant Self-Supervised Learning](https://mlsanity.com/paper/2608.21147/) (2026)
- [Phenomenon-Graph JEPA: Label-Efficient Representation Learning for Contactless Cardiorespiratory Sensing](https://mlsanity.com/paper/2609.32928/) (2026)
- [Pulseflow: PPG Counterfactual Generation Via Latent Transport](https://mlsanity.com/paper/2609.33501/) (2026)

## Abstract (arXiv)

Electrocardiography (ECG), photoplethysmography (PPG), and phonocardiography (PCG) provide complementary views of the same cardiac cycle, yet existing cardiac foundation models are trained for a single sensing modality, leaving the shared physiology across sensors unexploited. We introduce CardioState-JEPA, a cardiac foundation model to learn a single shared representation jointly across ECG, PPG, and PCG, built on a physiology-aware joint-embedding predictive architecture. The model maps heterogeneous waveforms into a common token space, processes them with a single shared Transformer encoder, and learns by predicting masked latent cardiac states, placing the pretraining target on shared physiology rather than sensor-specific waveform appearance. To handle the temporal offsets between electrical, mechanical, and hemodynamic events, cross-modal prediction uses a learned delay aligner that matches signals at the corresponding cardiac time. Because synchronized multi-sensor recordings are scarce, CardioState-JEPA first learns within-modality structure from abundant unimodal data and then uses paired data to align modalities in latent cardiac time. Evaluated as a frozen encoder across 25 downstream tasks spanning ECG, PPG, and PCG, our encoder improves average PPG classification by 8.2 AUROC points, PCG murmur detection by 18.8 AUROC points, and ECG classification by 15.5 AUROC points over the best self-supervised signal baseline and matches or exceeds cardiac models trained with privileged clinical text or supervised labels on several ECG benchmarks. These results establish that heterogeneous cardiac signals can mutually supervise a single foundation model of cardiac physiology.