mlsanityData to 14 Oct 2026

HiMAE: Hierarchical Masked Autoencoders Discover Resolution-Specific Structure in Wearable Time Series

arXiv 2510.25785 ↗First posted 28 Oct 2025cs.LG cs.AI eess.SPcited by 19

Not on the map. Why?

Abstract (arXiv)

Wearable sensors provide abundant physiological time series, yet the principles governing their predictive utility remain unclear. We hypothesize that temporal resolution is a fundamental axis of representation learning, with different clinical and behavioral outcomes relying on structure at distinct scales. To test this resolution hypothesis, we introduce HiMAE (Hierarchical Masked Autoencoder), a self supervised framework that combines masked autoencoding with a hierarchical convolutional encoder decoder. HiMAE produces multi resolution embeddings that enable systematic evaluation of which temporal scales carry predictive signal, transforming resolution from a hyperparameter into a probe for interpretability. Across classification, regression, and generative benchmarks, HiMAE consistently outperforms state of the art foundation models that collapse scale, while being orders of magnitude smaller. HiMAE is an efficient representation learner compact enough to run entirely on watch, achieving sub millisecond inference on smartwatch class CPUs for true edge inference. Together, these contributions position HiMAE as both an efficient self supervised learning method and a discovery tool for scale sensitive structure in wearable health.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 52 of its 104 references that are in the map. First posted on arXiv 28 Oct 2025.

Builds on

52 of 104 references.

References · 52

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  8. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  10. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  11. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  12. 2020Scaling Laws for Neural Language Modelscited by 6,586
38 more
  1. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  4. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  5. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  6. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  7. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  8. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 838
  9. 2020A Survey on Contrastive Self-supervised Learningcited by 554
  10. 2015An Introduction to Convolutional Neural Networkscited by 514
  11. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 378
  12. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 368
  13. 2022Masked Siamese Networks for Label-Efficient Learningcited by 302
  14. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 264
  15. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 239
  16. 2015Visualizing and Understanding Neural Models in NLPcited by 200
  17. 2022N-HiTS: Neural Hierarchical Interpolation for Time Series Forecastingcited by 146
  18. 2020Ethical Machine Learning in Health Carecited by 113
  19. 2024Pathformer: Multi-scale Transformers with Adaptive Pathways for Time Series Forecastingcited by 104
  20. 2023Large-scale Training of Foundation Models for Wearable Biosignalscited by 95
  21. 2022How Mask Matters: Towards Theoretical Understandings of Masked Autoencoderscited by 86
  22. 2024Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directionscited by 77
  23. 2022Self-supervised Learning for Human Activity Recognition Using 700,000 Person-days of Wearable Datacited by 75
  24. 2024PaPaGei: Open Foundation Models for Optical Physiological Signalscited by 69
  25. 2024Scaling Wearable Foundation Modelscited by 63
  26. 2022Scaleformer: Iterative Multi-scale Refining Transformers for Time Series Forecastingcited by 54
  27. 2024SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signalscited by 41
  28. 2023Understanding Masked Autoencoders via Hierarchical Latent Variable Modelscited by 39
  29. 2023How to estimate carbon footprint when training deep learning models? A guide and reviewcited by 34
  30. 2021Unraveling the Hidden Environmental Impacts of AI Solutions for Environmentcited by 32
  31. 2021Vision Transformers with Hierarchical Attentioncited by 31
  32. 2025LSM-2: Learning from Incomplete Wearable Sensor Datacited by 23
  33. 2024Can Large Language Models abstract Medical Coded Language?cited by 20
  34. 2025Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictionscited by 18
  35. 2024Text Serialization and Their Relationship with the Conventional Paradigms of Tabular Machine Learningcited by 17
  36. 2015Temporal Pyramid Pooling Based Convolutional Neural Networks for Action Recognitioncited by 15
  37. 2025A Case Study Exploring the Current Landscape of Synthetic Medical Record Generation with Commercial LLMscited by 13
  38. 2025Raptor: Scalable Train-Free Embeddings for 3D Medical Volumes Leveraging Pretrained 2D Foundation Modelscited by 13

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 19.

19 papers.

Sort:

By area:

  1. 2026SIGMA-PPG: Statistical-prior Informed Generative Masking Architecture for PPG Foundation Modelcited by 8
  2. 2026GlucoFM: A Dual-Stream Foundation Model for Continuous Glucose Monitoringcited by 7
  3. 2026Wavelet-Driven Masked Multiscale Reconstruction for PPG Foundation Modelscited by 5
  4. 2026Physiology-Aware Masked Cross-Modal Reconstruction for Biosignal Representation Learningcited by 5
  5. 2026Toward World Modeling of Physiological Signals with Chaos-Theoretic Balancing and Latent Dynamicscited by 3
  6. 2026SleepMaMi: A Universal Sleep Foundation Model for Integrating Macro- and Micro-structurescited by 2
  7. 2026Learning Clinical Representations Under Systematic Distribution Shiftcited by 2
  8. 2026Discriminative Representation Learning for Clinical Predictioncited by 2
  9. 2026Sonata: A Hybrid World Model for Inertial Kinematics under Clinical Data Scarcitycited by 2
  10. 2026Learning Longitudinal Health Representations from EHR and Wearable Datacited by 1
  11. 2026Uncertainty-Aware Foundation Models for Clinical Datacited by 1
  12. 2026Next-Token Prediction Learns Generalisable Representations of Sleep Physiologycited by 1
7 more
  1. 2026Dense Feature Learning via Linear Structure Preservation in Medical Datacited by 0
  2. 2026Event Fields: Learning Latent Event Structure for Waveform Foundation Modelscited by 0
  3. 2026WavesFM: Hierarchical Representation Learning for Longitudinal Wearable Sensor Waveformscited by 0
  4. 2026WISTERIA: Learning Clinical Representations from Noisy Supervision via Multi-View Consistency in Electronic Health Recordscited by 0
  5. 2026AURORA: Contextual Orthogonalization for Geometric Representation Learning in Healthcare Foundation Modelscited by 0
  6. 2026MorphologyFM: A Foundation Model for Morphology-Aware Representation Learning from ECG and Pulse Oximetry Waveformscited by 0
  7. 2026RAE-PPG: Duration-Grounded Retain-and-Extend Pretraining for PPG Foundation Modelscited by 0