Visualizing the Invisible: Generative Visual Grounding Empowers Universal EEG Understanding in MLLMs
Not on the map. Why?
Abstract (arXiv)
Leveraging the universal representations of pre-trained LLMs and MLLMs offers a promising path toward brain foundation models. However, visually-evoked EEG datasets remain scarce, leading existing methods to align neural signals mainly with abstract text, a lossy translation that may discard fine-grained perceptual information encoded in brain activity. We propose Generative Visual Grounding (GVG), a framework that visualizes the invisible by using an EEG-to-image generative model as a visual translator. Instead of forcing EEG into text alone, GVG hallucinates instance-specific proxy images for non-visual EEG, providing structured visual contexts that allow MLLMs to exploit their visual priors for clinical-state interpretation. We validate this idea on two MLLM backbones, GVG-X-Omni and GVG-Janus. Image-only alignment is already competitive: the lightweight GVG-X-Omni matches 1.7B-parameter text-aligned baselines while tuning only 170M parameters on a frozen 7B backbone. We further extend GVG-Janus with trimodal Image+Text alignment, where text supplies categorical semantic anchors and visual proxies enrich neural representations with perceptual details. Experiments show consistent gains in EEG understanding and visual generation, suggesting visual proxy grounding as an effective complement to textual alignment.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 55 references that are in the map. First posted on arXiv 18 May 2026.
Builds on
26 of 55 references.
References · 26
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
- 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
- 2024Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCIcited by 327
- 2022Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistencycited by 250
- 2021BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG datacited by 199
- 2024CBraMod: A Criss-Cross Brain Foundation Model for EEG Decodingcited by 199
- 2021Transformer-based Spatial-Temporal Feature Learning for EEG Decodingcited by 97
- 2024NeuroLM: A Universal Multi-task Foundation Model for Bridging the Gap between Language and EEG Signalscited by 88
- 2023BrainBERT: Self-supervised representation learning for intracranial recordingscited by 72
- 2023Cinematic Mindscapes: High-quality Video Reconstruction from Brain Activitycited by 71
14 more
- 2023DreamDiffusion: Generating High-Quality Images from Brain EEG Signalscited by 56
- 2020Inter-database validation of a deep learning approach for automatic sleep scoringcited by 38
- 2018Improved EEG Event Classification Using Differential Energycited by 32
- 2024EEG-GPT: Exploring Capabilities of Large Language Models for EEG Classification and Interpretationcited by 27
- 2023MBrain: A Multi-channel Self-Supervised Learning Framework for Brain Signalscited by 26
- 2025Scaling laws for decoding images from brain activitycited by 22
- 2023Brain Diffuser: An End-to-End Brain Image to Brain Network Pipelinecited by 21
- 2025CEReBrO: Compact Encoder for Representations of Brain Oscillations Using Efficient Alternating Attentioncited by 14
- 2025UniMind: Unleashing the Power of LLMs for Unified Multi-Task Brain Decodingcited by 14
- 2025NeurIPT: Foundation Model for Neural Interfacescited by 11
- 2025Uni-NTFM: A Unified Foundation Model for EEG Signal Representation Learningcited by 10
- 2025ECHO: Toward Contextual Seq2Seq Paradigms in Large EEG Modelscited by 7
- 2024BrainMAE: A Region-aware Self-supervised Learning Framework for Brain Signalscited by 6
- 2026Autoregressive Visual Decoding from EEG Signalscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Brain-to-Image Generation: Reconstructing Visual Stimuli from EEG using Generative Adversarial Networks
- 2026Do Models See in Line with Human Vision? Probing the Correspondence Between LVLM Representations and EEG Signals
- 2025Interpretable EEG-to-Image Generation with Semantic Prompts
- 2025CATVis: Context-Aware Thought Visualization
- 2025A Survey on Bridging EEG Signals and Generative AI: From Image and Text to Beyond
- 2024CognitionCapturer: Decoding Visual Stimuli From Human EEG Signal With Multimodal Information
- 2026EEG2Vision: A Multimodal EEG-Based Framework for 2D Visual Reconstruction in Cognitive Neuroscience
- 2026Brain3D: EEG-to-3D Decoding of Visual Representations via Multimodal Reasoning