mlsanityData to 7 Oct 2026

SleepMaMi: A Universal Sleep Foundation Model for Integrating Macro- and Micro-structures

arXiv 2602.07628 ↗First posted 7 Feb 2026cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

While the shift toward unified foundation models has revolutionized many deep learning domains, sleep medicine remains largely restricted to task-specific models that focus on localized micro-structure features. These approaches often neglect the rich, multi-modal context of Polysomnography (PSG) and fail to capture the global macro-structure of a full night's sleep. To address this, we introduce SleepMaMi , a Sleep Foundation Model engineered to master both hour-long sleep architectures and fine-grained signal morphologies. Our framework utilizes a hierarchical dual-encoder design: a Macro-Encoder to model full-night temporal dependencies and a Micro-Encoder to capture short-term characteristics from biosignals. Macro-Encoder is trained via Demographic-Guided Contrastive Learning, which aligns overnight sleep patterns with objective subject metadata, such as age, sex and BMI to refine global representations. Micro-Encoder is optimized via a hybrid Masked Autoencoder (MAE) and multi-modal contrastive objective. Pre-trained on a massive corpus of >20,000 PSG recordings (158K hours),SleepMaMi outperforms or matches state-of-the-art existing foundation models across a diverse suite of downstream tasks, demonstrating superior generalizability and label-efficient adaptation for clinical sleep analysis.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 30 of its 65 references that are in the map. First posted on arXiv 7 Feb 2026.

Builds on

30 of 65 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  7. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  9. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  10. 2020GLU Variants Improve Transformercited by 1,724
  11. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  12. 2019Root Mean Square Layer Normalizationcited by 1,425
18 more
  1. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  2. 2020On Layer Normalization in the Transformer Architecturecited by 906
  3. 2023A decoder-only foundation model for time-series forecastingcited by 650
  4. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  5. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
  6. 2016DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Networkcited by 354
  7. 2024Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCIcited by 311
  8. 2017DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEGcited by 154
  9. 2018Spreading vectors for similarity searchcited by 114
  10. 2024UniTS: A Unified Multi-Task Time Series Modelcited by 93
  11. 2018SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Stagingcited by 90
  12. 2020XSleepNet: Multi-View Sequential Model for Automatic Sleep Stagingcited by 87
  13. 2021SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantificationcited by 83
  14. 2022SimPer: Simple Self-Supervised Learning of Periodic Targetscited by 44
  15. 2024SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signalscited by 38
  16. 2019Dreem Open Datasets: Multi-Scored Sleep Datasets to compare Human and Automated sleep stagingcited by 33
  17. 2025HiMAE: Hierarchical Masked Autoencoders Discover Resolution-Specific Structure in Wearable Time Seriescited by 19
  18. 2022SleePyCo: Automatic Sleep Scoring with Feature Pyramid and Contrastive Learningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.