SleepMaMi: A Universal Sleep Foundation Model for Integrating Macro- and Micro-structures
Not on the map. Why?
Abstract (arXiv)
While the shift toward unified foundation models has revolutionized many deep learning domains, sleep medicine remains largely restricted to task-specific models that focus on localized micro-structure features. These approaches often neglect the rich, multi-modal context of Polysomnography (PSG) and fail to capture the global macro-structure of a full night's sleep. To address this, we introduce SleepMaMi , a Sleep Foundation Model engineered to master both hour-long sleep architectures and fine-grained signal morphologies. Our framework utilizes a hierarchical dual-encoder design: a Macro-Encoder to model full-night temporal dependencies and a Micro-Encoder to capture short-term characteristics from biosignals. Macro-Encoder is trained via Demographic-Guided Contrastive Learning, which aligns overnight sleep patterns with objective subject metadata, such as age, sex and BMI to refine global representations. Micro-Encoder is optimized via a hybrid Masked Autoencoder (MAE) and multi-modal contrastive objective. Pre-trained on a massive corpus of >20,000 PSG recordings (158K hours),SleepMaMi outperforms or matches state-of-the-art existing foundation models across a diverse suite of downstream tasks, demonstrating superior generalizability and label-efficient adaptation for clinical sleep analysis.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 30 of its 65 references that are in the map. First posted on arXiv 7 Feb 2026.
Builds on
30 of 65 references.
References · 30
- 2017Attention Is All You Needcited by 59,932
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
- 2020GLU Variants Improve Transformercited by 1,724
- 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
- 2019Root Mean Square Layer Normalizationcited by 1,425
18 more
- 2016Understanding intermediate layers using linear classifier probescited by 1,279
- 2020On Layer Normalization in the Transformer Architecturecited by 906
- 2023A decoder-only foundation model for time-series forecastingcited by 650
- 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
- 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
- 2016DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Networkcited by 354
- 2024Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCIcited by 311
- 2017DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEGcited by 154
- 2018Spreading vectors for similarity searchcited by 114
- 2024UniTS: A Unified Multi-Task Time Series Modelcited by 93
- 2018SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Stagingcited by 90
- 2020XSleepNet: Multi-View Sequential Model for Automatic Sleep Stagingcited by 87
- 2021SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantificationcited by 83
- 2022SimPer: Simple Self-Supervised Learning of Periodic Targetscited by 44
- 2024SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signalscited by 38
- 2019Dreem Open Datasets: Multi-Scored Sleep Datasets to compare Human and Automated sleep stagingcited by 33
- 2025HiMAE: Hierarchical Masked Autoencoders Discover Resolution-Specific Structure in Wearable Time Seriescited by 19
- 2022SleePyCo: Automatic Sleep Scoring with Feature Pyramid and Contrastive Learningcited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.