# Efficient Temporal Modeling for Mobile Sleep Staging via Lightweight Random Attention

arXiv: https://arxiv.org/abs/2606.13694 · HTML: https://mlsanity.com/paper/2606.13694/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 31 May 2026.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG](https://mlsanity.com/paper/1703.04046/) (2017)
- [SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Staging](https://mlsanity.com/paper/1809.10932/) (2018)
- [SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification](https://mlsanity.com/paper/2105.11043/) (2021)
- [L-SeqSleepNet: Whole-cycle Long Sequence Modelling for Automatic Sleep Staging](https://mlsanity.com/paper/2301.03441/) (2023)
- [Toward Interpretable Sleep Stage Classification Using Cross-Modal Transformers](https://mlsanity.com/paper/2208.06991/) (2022)
- [MSA-CNN: A Lightweight Multi-Scale CNN with Attention for Sleep Stage Classification](https://mlsanity.com/paper/2501.02949/) (2025)
- [Evaluating the Influence of Temporal Context on Automatic Mouse Sleep Staging through the Application of Human Models](https://mlsanity.com/paper/2406.16911/) (2024)
- [Sleep Modulation: The Challenge of Transitioning from Open Loop to Closed Loop](https://mlsanity.com/paper/2512.03784/) (2025)
- [eegFloss: A Python package for refining sleep EEG recordings using machine learning models](https://mlsanity.com/paper/2507.06433/) (2025)
- [ULW-SleepNet: An Ultra-Lightweight Network for Multimodal Sleep Stage Scoring](https://mlsanity.com/paper/2602.23852/) (2026)
- [Rethinking Random Transformers as Adaptive Sequence Smoothers for Sleep Staging](https://mlsanity.com/paper/2605.09905/) (2026)

## Abstract (arXiv)

Mobile sleep staging serves as a foundational infrastructure for in-home sleep monitoring and closed-loop modulation. But existing sequential models such as RNNs and Transformers are computationally expensive for mobile deployment. In this paper, we propose Random Attention (RA), a lightweight temporal modeling module based on fixed random projections, which replaces learnable sequence modeling with similarity-based aggregation. RA introduces little additional parameters beyond the epoch encoder while enabling effective temporal smoothing. We further provide a theoretical interpretation via the Random Attention Prior Kernel (RAPK), which decomposes RA into a global smoothing term and a feature similarity term, offering an interpretable view of temporal sleep structure. Experiments on Sleep-EDF-20 and Sleep-EDF-78 show that RA consistently improves epoch-wise baselines by 1-3% in accuracy and F1 score, while achieving competitive performance compared with LSTM, GRU, and Transformer models. RA also demonstrates strong generalization across different backbone encoders and improved robustness over conventional temporal smoothing methods. These results indicate that efficient sleep staging can be achieved through lightweight similarity-based temporal aggregation, making RA suitable for real-time wearable applications.