# sleep2vec: Unified Cross-Modal Alignment for Heterogeneous Nocturnal Biosignals

arXiv: https://arxiv.org/abs/2602.13857 · HTML: https://mlsanity.com/paper/2602.13857/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 57 references that are in the map.
- First posted on arXiv 14 Feb 2026.

## Builds on (most cited first, 18 of 18)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG](https://mlsanity.com/paper/1703.04046/) (2017)
- [SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Staging](https://mlsanity.com/paper/1809.10932/) (2018)
- [XSleepNet: Multi-View Sequential Model for Automatic Sleep Staging](https://mlsanity.com/paper/2007.05492/) (2020)
- [SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification](https://mlsanity.com/paper/2105.11043/) (2021)
- [PaPaGei: Open Foundation Models for Optical Physiological Signals](https://mlsanity.com/paper/2410.20542/) (2024)
- [Scaling Wearable Foundation Models](https://mlsanity.com/paper/2410.13638/) (2024)
- [SleepEEGNet: Automated Sleep Stage Scoring with Sequence to Sequence Deep Learning Approach](https://mlsanity.com/paper/1903.02108/) (2019)
- [Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals](https://mlsanity.com/paper/2412.09758/) (2024)
- [L-SeqSleepNet: Whole-cycle Long Sequence Modelling for Automatic Sleep Staging](https://mlsanity.com/paper/2301.03441/) (2023)
- [Generalizable Sleep Staging via Multi-Level Domain Alignment](https://mlsanity.com/paper/2401.05363/) (2023)
- [Automatic sleep stage classification with deep residual networks in a mixed-cohort setting](https://mlsanity.com/paper/2008.09416/) (2020)
- [Promoting cross-modal representations to improve multimodal foundation models for physiological signals](https://mlsanity.com/paper/2410.16424/) (2024)
- [Sleep Staging from Electrocardiography and Respiration with Deep Learning](https://mlsanity.com/paper/1908.11463/) (2019)
- [SleepVST: Sleep Staging from Near-Infrared Video Signals using Pre-Trained Transformers](https://mlsanity.com/paper/2404.03831/) (2024)

## Built on by (most cited first, 4 of 4)

- [Next-Token Prediction Learns Generalisable Representations of Sleep Physiology](https://mlsanity.com/paper/2606.09605/) (2026)
- [Physiological World Models for Human State Transitions](https://mlsanity.com/paper/2608.15309/) (2026)
- [SemPSG: A Semantic Channel-Aware Foundation Model for Polysomnography Analysis](https://mlsanity.com/paper/2609.36619/) (2026)
- [Training and Scaling Compute-Optimal Physiological Waveform Foundation Models](https://mlsanity.com/paper/2610.05649/) (2026)

## Abstract (arXiv)

Tasks ranging from sleep staging to clinical diagnosis traditionally rely on standard polysomnography (PSG) devices, bedside monitors and wearable devices, which capture diverse nocturnal biosignals (e.g., EEG, EOG, ECG, SpO2). However, heterogeneity across devices and frequent sensor dropout pose significant challenges for unified modelling of these multimodal signals. We present textttsleep2vec, a foundation model for diverse and incomplete nocturnal biosignals that learns a shared representation via cross-modal alignment. textttsleep2vec is contrastively pre-trained on 42,249 overnight recordings spanning nine modalities using a textitDemography, Age, Site & History-aware InfoNCE objective that incorporates physiological and acquisition metadata (textite.g., age, gender, recording site) to dynamically weight negatives and mitigate cohort-specific shortcuts. On downstream sleep staging and clinical outcome assessment, textttsleep2vec consistently outperforms strong baselines and remains robust to any subset of available modalities and sensor dropout. We further characterize, to our knowledge for the first time, scaling laws for nocturnal biosignals with respect to modality diversity and model capacity. Together, these results show that unified cross-modal alignment, coupled with principled scaling, enables label-efficient, general-purpose modelling of real-world nocturnal biosignals.