mlsanityData to 7 Oct 2026

A Language-Guided Multimodal Foundation Model for Zero-Shot and Multi-Task Brain Signal Analysis

arXiv 2609.15740 ↗First posted 14 Sep 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Brain signal analysis is essential for both neuroscience research and clinical diagnostics, yet current approaches face critical limitations. End-to-end models require task-specific retraining and exhibit limited generalization, while pre-trained models lack semantic depth and still depend on extensive fine-tuning. Meanwhile, general-purpose multimodal foundation models, though powerful in other domains, struggle to interpret brain signals due to representational misalignment and lack of domain knowledge. This study introduces a multimodal foundation model for zero-shot and multi-task brain signal analysis (METIS) through a unified language-signal alignment framework. METIS is pretrained on the largest and most diverse brain-signal corpus to date, comprising over 70,000 h of recordings from more than 11,000 subjects across 20 datasets. In a comprehensive zero-shot evaluation across 12 datasets, METIS outperformed the leading generalist model by over 20.9% in average accuracy. Remarkably, without any fine-tuning, METIS's performance matches or exceeds that of supervised, task-specific models. Furthermore, METIS demonstrates exceptional data efficiency and strong generalization, achieving an average AUROC advantage of over 16.0% in few-shot settings and 15.9% in cross-dataset transfer. This work establishes a new paradigm for general-purpose brain signal analysis, paving the way for next-generation neurotechnology.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 83 references that are in the map. First posted on arXiv 14 Sep 2026.

Builds on

26 of 83 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  8. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  9. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  10. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  11. 2024Qwen2.5 Technical Reportcited by 4,713
  12. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
14 more
  1. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  3. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  4. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  5. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  6. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  7. 2016EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfacescited by 616
  8. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  9. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 559
  10. 2024Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCIcited by 311
  11. 2024CBraMod: A Criss-Cross Brain Foundation Model for EEG Decodingcited by 188
  12. 2018SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Stagingcited by 90
  13. 2018The Temple University Hospital Seizure Detection Corpuscited by 80
  14. 2018Improved EEG Event Classification Using Differential Energycited by 32

Built on by

Not cited yet.