# BiTimeCrossNet: Time-Aware Self-Supervised Learning for Pediatric Sleep

arXiv: https://arxiv.org/abs/2602.02769 · HTML: https://mlsanity.com/paper/2602.02769/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 41 references that are in the map.
- First posted on arXiv 2 Feb 2026.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free](https://mlsanity.com/paper/2505.06708/) (2025)
- [BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data](https://mlsanity.com/paper/2101.12037/) (2021)
- [SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signals](https://mlsanity.com/paper/2405.17766/) (2024)
- [A Large Collection of Real-world Pediatric Sleep Studies](https://mlsanity.com/paper/2102.13284/) (2021)
- [Positional Encoding in Transformer-Based Time Series Models: A Survey](https://mlsanity.com/paper/2502.12370/) (2025)
- [NeuroNet: A Novel Hybrid Self-Supervised Learning Framework for Sleep Stage Classification Using Single-Channel EEG](https://mlsanity.com/paper/2404.17585/) (2024)
- [Automatic Sleep Scoring from Large-scale Multi-channel Pediatric EEG](https://mlsanity.com/paper/2207.06921/) (2022)
- [wav2sleep: A Unified Multi-Modal Approach to Sleep Stage Classification from Physiological Signals](https://mlsanity.com/paper/2411.04644/) (2024)
- [PedSleepMAE: Generative Model for Multimodal Pediatric Sleep Signals](https://mlsanity.com/paper/2411.00718/) (2024)
- [STAMP: Spatial-Temporal Adapter with Multi-Head Pooling](https://mlsanity.com/paper/2511.10848/) (2025)

## Built on by (most cited first, 1 of 1)

- [Learning transferable human physiology from two million hours of sleep with SleepFM-2](https://mlsanity.com/paper/2609.06849/) (2026)

## Abstract (arXiv)

We present BiTimeCrossNet (BTCNet), a multimodal self-supervised learning framework for long physiological recordings such as overnight sleep studies. While many existing approaches train on short segments treated as independent samples, BTCNet incorporates information about when each segment occurs within its parent recording, for example within a sleep session. BTCNet further learns pairwise interactions between physiological signals via cross-attention, without requiring task labels or sequence-level supervision. We evaluate BTCNet on pediatric sleep data across six downstream tasks, including sleep staging, arousal detection, and respiratory event detection. Under frozen-backbone linear probing, BTCNet consistently outperforms an otherwise identical non-time-aware variant, with gains that generalize to an independent pediatric dataset. Compared to existing multimodal self-supervised sleep models, BTCNet achieves strong performance, particularly on respiration-related tasks.