Towards Robust Multimodal Physiological Foundation Models: Handling Arbitrary Missing Modalities
Not on the map. Why?
Abstract (arXiv)
Multimodal physiological signals, such as EEG, ECG, EOG, and EMG, are crucial for healthcare and brain-computer interfaces. While existing methods rely on specialized architectures and dataset-specific fusion strategies, they struggle to learn universal representations that generalize across datasets and handle missing modalities at inference time. To address these issues, we propose PhysioOmni, a foundation model for multimodal physiological signal analysis that models both homogeneous and heterogeneous features to decouple multimodal signals and extract generic representations while maintaining compatibility with arbitrary missing modalities. PhysioOmni trains a decoupled multimodal tokenizer, enabling masked signal pre-training via modality-invariant and modality-specific objectives. To ensure adaptability to diverse and incomplete modality combinations, the pre-trained encoders undergo resilient fine-tuning with prototype alignment on downstream datasets. Extensive experiments on four downstream tasks, emotion recognition, sleep stage classification, motor prediction, and mental workload detection, demonstrate that PhysioOmni achieves state-of-the-art performance while maintaining strong robustness to missing modalities. Our code and model weights will be released.
Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 20 of its 46 references that are in the map. First posted on arXiv 28 Apr 2025.
Builds on
20 of 46 references.
References · 20
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2017Neural Discrete Representation Learningcited by 4,712
- 2016Gaussian Error Linear Units (GELUs)cited by 3,502
- 2020GLU Variants Improve Transformercited by 1,804
- 2018Group Normalizationcited by 1,524
- 2019Root Mean Square Layer Normalizationcited by 1,480
- 2024Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCIcited by 327
- 2020MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysiscited by 265
- 2024CBraMod: A Criss-Cross Brain Foundation Model for EEG Decodingcited by 199
- 2024NeuroLM: A Universal Multi-task Foundation Model for Bridging the Gap between Language and EEG Signalscited by 88
8 more
- 2023Decoding Natural Images from EEG for Object Recognitioncited by 81
- 2023Decoupled Multimodal Distilling for Emotion Recognitioncited by 76
- 2017A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time seriescited by 58
- 2020Inter-database validation of a deep learning approach for automatic sleep scoringcited by 38
- 2025Reading Your Heart: Learning ECG Words and Sentences via Pre-training ECG Language Modelcited by 36
- 2024Brant-X: A Unified Physiological Signal Alignment Frameworkcited by 17
- 2024Promoting cross-modal representations to improve multimodal foundation models for physiological signalscited by 13
- 2023Learning Unseen Modality Interactioncited by 12
Built on by
Citing papers per year: 2025: 2; 2026: 9.
11 papers.
- 2025TinyMyo: a Tiny Foundation Model for Flexible EMG Signal Processing at the Edgecited by 7
- 2026Wearable Foundation Models Should Go Beyond Static Encoderscited by 3
- 2025LEAF: Language-EEG Aligned Foundation Model for Brain-Computer Interfacescited by 2
- 2026Brain-OF: An Omnifunctional Foundation Model for fMRI, EEG and MEGcited by 2
- 2026Compact Latent Manifold Translation: A Parameter-Efficient Foundation Model for Cross-Modal and Cross-Frequency Physiological Signal Synthesiscited by 1
- 2026PanLUNA: An Efficient and Robust Query-Unified Multimodal Model for Edge Biosignal Intelligencecited by 0
- 2026Sense Less, Infer More: Agentic Multimodal Transformers for Edge Medical Intelligencecited by 0
- 2026PAMF: Prior-Aware Multimodal Fusion for Incomplete Time Series Datacited by 0
- 2026EEG-PRIME: Prototype-Aligned Representation Learning with Multi-Level Conditioning for EEG Decodingcited by 0
- 2026EEGBind: Detecting Source-Level Interictal Epileptiform Discharges via EEG-Centric Multimodal Bindingcited by 0
- 2026ZeroMAG: Zero-Shot Multimodal Adapter Generation for Plug-and-Play EEG Foundation Modelscited by 0
- 2026ZeroMAG: Zero-Shot Multimodal Adapter Generation for Plug-and-Play EEG Foundation Models2 Oct 2026
- 2026EEGBind: Detecting Source-Level Interictal Epileptiform Discharges via EEG-Centric Multimodal Binding9 Sep 2026
- 2026EEG-PRIME: Prototype-Aligned Representation Learning with Multi-Level Conditioning for EEG Decoding13 Aug 2026
- 2026PAMF: Prior-Aware Multimodal Fusion for Incomplete Time Series Data4 Jun 2026
- 2026Compact Latent Manifold Translation: A Parameter-Efficient Foundation Model for Cross-Modal and Cross-Frequency Physiological Signal Synthesis13 May 2026
- 2026Sense Less, Infer More: Agentic Multimodal Transformers for Edge Medical Intelligence12 Apr 2026
- 2026PanLUNA: An Efficient and Robust Query-Unified Multimodal Model for Edge Biosignal Intelligence5 Apr 2026
- 2026Wearable Foundation Models Should Go Beyond Static Encoders20 Mar 2026
- 2026Brain-OF: An Omnifunctional Foundation Model for fMRI, EEG and MEG26 Feb 2026
- 2025TinyMyo: a Tiny Foundation Model for Flexible EMG Signal Processing at the Edge5 Dec 2025
- 2025LEAF: Language-EEG Aligned Foundation Model for Brain-Computer Interfaces29 Sep 2025