# Multimodal LLMs Can Learn to Read Brain Signals: A Vision--Language Model for Unified Multi-Task EEG Decoding

arXiv: https://arxiv.org/abs/2610.09355 · HTML: https://mlsanity.com/paper/2610.09355/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 13 of its 63 references that are in the map.
- First posted on arXiv 7 Oct 2026.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 13 of 13)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [Transformer-based Spatial-Temporal Feature Learning for EEG Decoding](https://mlsanity.com/paper/2106.11170/) (2021)
- [REVE: A Foundation Model for EEG -- Adapting to Any Setup with Large-Scale Pretraining on 25,000 Subjects](https://mlsanity.com/paper/2510.21585/) (2025)
- [Self-supervised EEG Representation Learning for Automatic Sleep Staging](https://mlsanity.com/paper/2110.15278/) (2021)
- [EEG-GPT: Exploring Capabilities of Large Language Models for EEG Classification and Interpretation](https://mlsanity.com/paper/2401.18006/) (2024)
- [TimeMaster: Training Time-Series Multimodal LLMs to Reason via Reinforcement Learning](https://mlsanity.com/paper/2506.13705/) (2025)
- [UniMind: Unleashing the Power of LLMs for Unified Multi-Task Brain Decoding](https://mlsanity.com/paper/2506.18962/) (2025)
- [VISTA: Vision-Language Inference for Training-Free Stock Time-Series Analysis](https://mlsanity.com/paper/2505.18570/) (2025)
- [WaveMind: Towards a Conversational EEG Foundation Model Aligned to Textual and Visual Modalities](https://mlsanity.com/paper/2510.00032/) (2025)
- [NeuroMambaLLM: Dynamic Graph Learning of fMRI Functional Connectivity in Autistic Brains Using Mamba and Language Model Reasoning](https://mlsanity.com/paper/2602.13770/) (2026)
- [Functional embeddings enable Aggregation of multi-area SEEG recordings over subjects and sessions](https://mlsanity.com/paper/2510.27090/) (2025)

## Abstract (arXiv)

Learning EEG representations that generalize across cognitive tasks, subjects, and recording conditions remains a key challenge in electroencephalography (EEG) decoding. Recent advances in foundation models have improved EEG decoding performance, yet a fundamental open question remains: how to effectively interface neural signals with these models to enable multi-task learning across datasets. To investigate this question, we introduce BraVista, a visual-language framework that encodes multichannel EEG signals as structured images and enables multi-task learning through instruction-conditioned vision-language models (VLMs). Our approach relies on continued post-training of a general-domain VLM, leveraging its visual and linguistic priors to adapt to neural signals without a separate large-scale EEG-specific pretraining stage. We evaluate BraVista on four datasets spanning sleep staging, emotion recognition, cognitive workload classification, and abnormal EEG detection, showing strong performance across these tasks. Further analyses show that the choice of EEG-to-image representation is critical to performance. Moreover, through controlled perturbations of the EEG signal, we observe a gradual performance degradation under increasing noise, suggesting that the model relies on EEG-relevant information rather than superficial visual patterns. Together, these findings establish structured visual representations as an effective and scalable interface between neural signals and general-domain foundation models for unified multi-task EEG decoding.