mlsanityData to 14 Oct 2026

Towards Robust Multimodal Physiological Foundation Models: Handling Arbitrary Missing Modalities

arXiv 2504.19596 ↗First posted 28 Apr 2025eess.SP cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Multimodal physiological signals, such as EEG, ECG, EOG, and EMG, are crucial for healthcare and brain-computer interfaces. While existing methods rely on specialized architectures and dataset-specific fusion strategies, they struggle to learn universal representations that generalize across datasets and handle missing modalities at inference time. To address these issues, we propose PhysioOmni, a foundation model for multimodal physiological signal analysis that models both homogeneous and heterogeneous features to decouple multimodal signals and extract generic representations while maintaining compatibility with arbitrary missing modalities. PhysioOmni trains a decoupled multimodal tokenizer, enabling masked signal pre-training via modality-invariant and modality-specific objectives. To ensure adaptability to diverse and incomplete modality combinations, the pre-trained encoders undergo resilient fine-tuning with prototype alignment on downstream datasets. Extensive experiments on four downstream tasks, emotion recognition, sleep stage classification, motor prediction, and mental workload detection, demonstrate that PhysioOmni achieves state-of-the-art performance while maintaining strong robustness to missing modalities. Our code and model weights will be released.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 20 of its 46 references that are in the map. First posted on arXiv 28 Apr 2025.

Builds on

20 of 46 references.

References · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2017Neural Discrete Representation Learningcited by 4,712
  5. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  6. 2020GLU Variants Improve Transformercited by 1,804
  7. 2018Group Normalizationcited by 1,524
  8. 2019Root Mean Square Layer Normalizationcited by 1,480
  9. 2024Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCIcited by 327
  10. 2020MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysiscited by 265
  11. 2024CBraMod: A Criss-Cross Brain Foundation Model for EEG Decodingcited by 199
  12. 2024NeuroLM: A Universal Multi-task Foundation Model for Bridging the Gap between Language and EEG Signalscited by 88
8 more
  1. 2023Decoding Natural Images from EEG for Object Recognitioncited by 81
  2. 2023Decoupled Multimodal Distilling for Emotion Recognitioncited by 76
  3. 2017A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time seriescited by 58
  4. 2020Inter-database validation of a deep learning approach for automatic sleep scoringcited by 38
  5. 2025Reading Your Heart: Learning ECG Words and Sentences via Pre-training ECG Language Modelcited by 36
  6. 2024Brant-X: A Unified Physiological Signal Alignment Frameworkcited by 17
  7. 2024Promoting cross-modal representations to improve multimodal foundation models for physiological signalscited by 13
  8. 2023Learning Unseen Modality Interactioncited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 9.

11 papers.