# Neural-MCRL: Neural Multimodal Contrastive Representation Learning for EEG-based Visual Decoding

arXiv: https://arxiv.org/abs/2412.17337 · HTML: https://mlsanity.com/paper/2412.17337/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 29 references that are in the map.
- First posted on arXiv 23 Dec 2024.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Decoupled Contrastive Learning](https://mlsanity.com/paper/2110.06848/) (2021)
- [Decoding Natural Images from EEG for Object Recognition](https://mlsanity.com/paper/2308.13234/) (2023)
- [Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion](https://mlsanity.com/paper/2403.07721/) (2024)
- [Cinematic Mindscapes: High-quality Video Reconstruction from Brain Activity](https://mlsanity.com/paper/2305.11675/) (2023)
- [Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Features](https://mlsanity.com/paper/2210.06756/) (2022)
- [Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning](https://mlsanity.com/paper/2303.05952/) (2023)
- [Semi-supervised Semantic Segmentation with Prototype-based Consistency Regularization](https://mlsanity.com/paper/2210.04388/) (2022)
- [Object classification from randomized EEG trials](https://mlsanity.com/paper/2004.06046/) (2020)
- [A Tale of Single-channel Electroencephalogram: Devices, Datasets, Signal Processing, Applications, and Future Directions](https://mlsanity.com/paper/2407.14850/) (2024)
- [You Only Acquire Sparse-channel (YOAS): A Unified Framework for Dense-channel EEG Generation](https://mlsanity.com/paper/2406.15269/) (2024)

## Built on by (most cited first, 14 of 14)

- [NeuroBridge: Bio-Inspired Self-Supervised EEG-to-Image Decoding via Cognitive Priors and Bidirectional Semantic Alignment](https://mlsanity.com/paper/2511.06836/) (2025)
- [Aligning What EEG Can See: Structural Representations for Brain-Vision Matching](https://mlsanity.com/paper/2603.07077/) (2026)
- [Foundation Models for Cross-Domain EEG Analysis Application: A Survey](https://mlsanity.com/paper/2508.15716/) (2025)
- [Deep Models, Shallow Alignment: Uncovering the Granularity Mismatch in Neural Decoding](https://mlsanity.com/paper/2601.21948/) (2026)
- [Subject-Aware Multi-Granularity Alignment for Zero-Shot EEG-to-Image Retrieval](https://mlsanity.com/paper/2604.17782/) (2026)
- [FreqDGT: Frequency-Adaptive Dynamic Graph Networks with Transformer for Cross-subject EEG Emotion Recognition](https://mlsanity.com/paper/2506.22807/) (2025)
- [LEL: Lipschitz Continuity Constrained Ensemble Learning for Efficient EEG-Based Intra-subject Emotion Recognition](https://mlsanity.com/paper/2504.09156/) (2025)
- [Multi-Level Bidirectional Biomimetic Learning for EEG-Based Visual Decoding](https://mlsanity.com/paper/2605.04680/) (2026)
- [STAMBRIDGE: Spectral-Temporal Amplitude-aware Mid-Feature Bridge for EEG Visual Decoding](https://mlsanity.com/paper/2605.23137/) (2026)
- [Region-aware Spatiotemporal Modeling with Collaborative Domain Generalization for Cross-Subject EEG Emotion Recognition](https://mlsanity.com/paper/2601.15615/) (2026)
- [Neural Visual Decoding via Cognitive guided Adaptive Blurring and Information Constrained Alignment](https://mlsanity.com/paper/2605.16418/) (2026)
- [MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding](https://mlsanity.com/paper/2605.24523/) (2026)
- [SUP-MCRL: Subject-aware Unified Pseudo-feature Coded Multimodal Contrastive Representation Learning for EEG Visual Decoding](https://mlsanity.com/paper/2606.16615/) (2026)
- [FSDBN: Foreground-Aware EEG-Visual Alignment via Dynamic Brain Networks](https://mlsanity.com/paper/2607.18344/) (2026)

## Abstract (arXiv)

Decoding neural visual representations from electroencephalogram (EEG)-based brain activity is crucial for advancing brain-machine interfaces (BMI) and has transformative potential for neural sensory rehabilitation. While multimodal contrastive representation learning (MCRL) has shown promise in neural decoding, existing methods often overlook semantic consistency and completeness within modalities and lack effective semantic alignment across modalities. This limits their ability to capture the complex representations of visual neural responses. We propose Neural-MCRL, a novel framework that achieves multimodal alignment through semantic bridging and cross-attention mechanisms, while ensuring completeness within modalities and consistency across modalities. Our framework also features the Neural Encoder with Spectral-Temporal Adaptation (NESTA), a EEG encoder that adaptively captures spectral patterns and learns subject-specific transformations. Experimental results demonstrate significant improvements in visual decoding accuracy and model generalization compared to state-of-the-art methods, advancing the field of EEG-based neural visual representation decoding in BMI. Codes will be available at: https://github.com/NZWANG/Neural-MCRL.