A Language-Guided Multimodal Foundation Model for Zero-Shot and Multi-Task Brain Signal Analysis
Not on the map. Why?
Abstract (arXiv)
Brain signal analysis is essential for both neuroscience research and clinical diagnostics, yet current approaches face critical limitations. End-to-end models require task-specific retraining and exhibit limited generalization, while pre-trained models lack semantic depth and still depend on extensive fine-tuning. Meanwhile, general-purpose multimodal foundation models, though powerful in other domains, struggle to interpret brain signals due to representational misalignment and lack of domain knowledge. This study introduces a multimodal foundation model for zero-shot and multi-task brain signal analysis (METIS) through a unified language-signal alignment framework. METIS is pretrained on the largest and most diverse brain-signal corpus to date, comprising over 70,000 h of recordings from more than 11,000 subjects across 20 datasets. In a comprehensive zero-shot evaluation across 12 datasets, METIS outperformed the leading generalist model by over 20.9% in average accuracy. Remarkably, without any fine-tuning, METIS's performance matches or exceeds that of supervised, task-specific models. Furthermore, METIS demonstrates exceptional data efficiency and strong generalization, achieving an average AUROC advantage of over 16.0% in few-shot settings and 15.9% in cross-dataset transfer. This work establishes a new paradigm for general-purpose brain signal analysis, paving the way for next-generation neurotechnology.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 83 references that are in the map. First posted on arXiv 14 Sep 2026.
Builds on
26 of 83 references.
References · 26
- 2017Attention Is All You Needcited by 59,932
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023GPT-4 Technical Reportcited by 18,132
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2023Visual Instruction Tuningcited by 7,792
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
14 more
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
- 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
- 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
- 2016EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfacescited by 616
- 2022Mixture-of-Experts with Expert Choice Routingcited by 611
- 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 559
- 2024Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCIcited by 311
- 2024CBraMod: A Criss-Cross Brain Foundation Model for EEG Decodingcited by 188
- 2018SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Stagingcited by 90
- 2018The Temple University Hospital Seizure Detection Corpuscited by 80
- 2018Improved EEG Event Classification Using Differential Energycited by 32
Built on by
Not cited yet.