# Semantic-aware Contrastive Learning for Electroencephalography-to-Text Generation with Curriculum Learning

arXiv: https://arxiv.org/abs/2301.09237 · HTML: https://mlsanity.com/paper/2301.09237/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 41 references that are in the map.
- First posted on arXiv 23 Jan 2023.

## Builds on (most cited first, 14 of 14)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://mlsanity.com/paper/2006.03659/) (2020)
- [Subject-Aware Contrastive Learning for Biosignals](https://mlsanity.com/paper/2007.04871/) (2020)
- [Open Vocabulary Electroencephalography-To-Text Decoding and Zero-shot Sentiment Classification](https://mlsanity.com/paper/2112.02690/) (2021)
- [ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotation](https://mlsanity.com/paper/1912.00903/) (2019)
- [Toward a realistic model of speech processing in the brain with self-supervised learning](https://mlsanity.com/paper/2206.01685/) (2022)
- [Contrastive Learning of Subject-Invariant EEG Representations for Cross-Subject Emotion Recognition](https://mlsanity.com/paper/2109.09559/) (2021)
- [Inter-subject Contrastive Learning for Subject Adaptive EEG-based Visual Recognition](https://mlsanity.com/paper/2202.02901/) (2022)

## Built on by (most cited first, 5 of 5)

- [Deep Representation Learning for Open Vocabulary Electroencephalography-to-Text Decoding](https://mlsanity.com/paper/2312.09430/) (2023)
- [Bridging Brain Signals and Language: A Deep Learning Approach to EEG-to-Text Decoding](https://mlsanity.com/paper/2502.17465/) (2025)
- [Bridging Brain with Foundation Models through Self-Supervised Learning](https://mlsanity.com/paper/2506.16009/) (2025)
- [Spatial-Temporal Transformer with Curriculum Learning for EEG-Based Emotion Recognition](https://mlsanity.com/paper/2507.14698/) (2025)
- [Neurocognitive Modeling for Text Generation: Deep Learning Architecture for EEG Data](https://mlsanity.com/paper/2509.07202/) (2025)

## Abstract (arXiv)

Electroencephalography-to-Text generation (EEG-to-Text), which aims to directly generate natural text from EEG signals has drawn increasing attention in recent years due to the enormous potential for Brain-computer interfaces (BCIs). However, the remarkable discrepancy between the subject-dependent EEG representation and the semantic-dependent text representation poses a great challenge to this task. To mitigate this challenge, we devise a Curriculum Semantic-aware Contrastive Learning strategy (C-SCL), which effectively re-calibrates the subject-dependent EEG representation to the semantic-dependent EEG representation, thus reducing the discrepancy. Specifically, our C-SCL pulls semantically similar EEG representations together while pushing apart dissimilar ones. Besides, in order to introduce more meaningful contrastive pairs, we carefully employ curriculum learning to not only craft meaningful contrastive pairs but also make the learning progressively. We conduct extensive experiments on the ZuCo benchmark and our method combined with diverse models and architectures shows stable improvements across three types of metrics while achieving the new state-of-the-art. Further investigation proves not only its superiority in both the single-subject and low-resource settings but also its robust generalizability in the zero-shot setting.