# EEG-CLIP : Learning EEG representations from natural language descriptions

arXiv: https://arxiv.org/abs/2503.16531 · HTML: https://mlsanity.com/paper/2503.16531/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 14 references that are in the map.
- First posted on arXiv 18 Mar 2025.

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://mlsanity.com/paper/2210.10341/) (2022)
- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [Deep learning-based electroencephalography analysis: a systematic review](https://mlsanity.com/paper/1901.05498/) (2019)
- [Brain-to-Text Decoding: A Non-invasive Approach via Typing](https://mlsanity.com/paper/2502.17480/) (2025)
- [Deep learning with convolutional neural networks for decoding and visualization of EEG pathology](https://mlsanity.com/paper/1708.08012/) (2017)
- [A large-scale evaluation framework for EEG deep learning architectures](https://mlsanity.com/paper/1806.07741/) (2018)

## Built on by (most cited first, 9 of 9)

- [Large Language Models for EEG: A Comprehensive Survey and Taxonomy](https://mlsanity.com/paper/2506.06353/) (2025)
- [Foundation Models for Cross-Domain EEG Analysis Application: A Survey](https://mlsanity.com/paper/2508.15716/) (2025)
- [Neural Signals Generate Clinical Notes in the Wild](https://mlsanity.com/paper/2601.22197/) (2026)
- [Learning Transferable Sensor Models via Language-Informed Pretraining](https://mlsanity.com/paper/2603.11950/) (2026)
- [CAMEL-CLIP: Channel-aware Multimodal Electroencephalography-text Alignment for Generalizable Brain Foundation Models](https://mlsanity.com/paper/2603.13272/) (2026)
- [NeuroNarrator: A Generalist EEG-to-Text Foundation Model for Clinical Interpretation via Spectro-Spatial Grounding and Temporal State-Space Reasoning](https://mlsanity.com/paper/2603.16880/) (2026)
- [MyoSem: Aligning Electromyography with Natural-Language Action Semantics](https://mlsanity.com/paper/2606.00174/) (2026)
- [EVA-Net: Subject-Independent EEG Motor Decoding with Video-Derived Motor Priors](https://mlsanity.com/paper/2606.01884/) (2026)
- [MindAU: EEG-Conditioned Facial Action Unit Editing via Dual-Stream Manifold Alignment](https://mlsanity.com/paper/2607.00410/) (2026)

## Abstract (arXiv)

Deep networks for electroencephalogram (EEG) decoding are often only trained to solve one specific task, such as pathology or age decoding. A more general task-agnostic approach is to train deep networks to match a (clinical) EEG recording to its corresponding textual medical report and vice versa. This approach was pioneered in the computer vision domain matching images and their text captions and subsequently allowed to do successful zero-shot decoding using textual class prompts. In this work, we follow this approach and develop a contrastive learning framework, EEG-CLIP, that aligns the EEG time series and the descriptions of the corresponding clinical text in a shared embedding space. We investigated its potential for versatile EEG decoding, evaluating performance in a range of few-shot and zero-shot settings. Overall, we show that EEG-CLIP manages to non-trivially align text and EEG representations. Our work presents a promising approach to learn general EEG representations, which could enable easier analyses of diverse decoding questions through zero-shot decoding or training task-specific models from fewer training examples. The code for reproducing our results is available at https://github.com/tidiane-camaret/EEGClip