mlsanityData to 7 Oct 2026

Mind's Eye: Image Recognition by EEG via Multimodal Similarity-Keeping Contrastive Learning

arXiv 2406.16910 ↗First posted 5 Jun 2024eess.SP cs.AI cs.HCcited by 12

Not on the map. Why?

Abstract (arXiv)

Decoding images from non-invasive electroencephalographic (EEG) signals has been a grand challenge in understanding how the human brain process visual information in real-world scenarios. To cope with the issues of signal-to-noise ratio and nonstationarity, this paper introduces a MUltimodal Similarity-keeping contrastivE learning (MUSE) framework for zero-shot EEG-based image classification. We develop a series of multivariate time-series encoders tailored for EEG signals and assess the efficacy of regularized contrastive EEG-Image pretraining using an extensive visual EEG dataset. Our method achieves state-of-the-art performance, with a top-1 accuracy of 19.3% and a top-5 accuracy of 48.8% in 200-way zero-shot image classification. Furthermore, we visualize neural patterns via model interpretation, shedding light on the visual processing dynamics in the human brain. The code repository for this work is available at: https://github.com/ChiShengChen/MUSE_EEG.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 19 of its 49 references that are in the map. First posted on arXiv 5 Jun 2024.

Builds on

19 of 49 references.

References · 19

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2017Graph Attention Networkscited by 8,854
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2019Contrastive Multiview Codingcited by 1,297
  9. 2021How Attentive are Graph Attention Networks?cited by 818
  10. 2024MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Datacited by 105
  11. 2023Decoding Natural Images from EEG for Object Recognitioncited by 79
  12. 2016Deep Learning Human Mind for Automated Visual Classificationcited by 76
7 more
  1. 2022Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Featurescited by 63
  2. 2022MAtt: A Manifold Attention Network for EEG Decodingcited by 43
  3. 2023Learning Robust Deep Visual Representations from EEG Brain Recordingscited by 26
  4. 2020Object classification from randomized EEG trialscited by 19
  5. 2023Brain-Conditional Multimodal Synthesis: A Survey and Taxonomycited by 15
  6. 2022See What You See: Self-supervised Cross-modal Retrieval of Visual Stimuli from Brain Activitycited by 4
  7. 2023SSVEP-DAN: A Data Alignment Network for SSVEP-based Brain Computer Interfacescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 8; 2026: 3.

12 papers.

  1. 2025Quantum Adaptive Self-Attention for Quantum Transformer Modelscited by 14
  2. 2024NECOMIMI: Neural-Cognitive Multimodal EEG-informed Image Generation with Diffusion Modelscited by 10
  3. 2025Large Cognition Model: Towards Pretrained EEG Foundation Modelcited by 10
  4. 2025Quantum-Enhanced Natural Language Generation: A Multi-Model Framework with Hybrid Quantum-Classical Architecturescited by 4
  5. 2025Exploring the Potential of QEEGNet for Cross-Task and Cross-Dataset Electroencephalography Encoding with Quantum Machine Learningcited by 3
  6. 2025EEG2TEXT-CN: An Exploratory Study of Open-Vocabulary Chinese Text-EEG Alignment via Large Language Model and Contrastive Learning on ChineseEEGcited by 3
  7. 2025Optimal Transport for Brain-Image Alignment: Unveiling Redundancy and Synergy in Neural Information Processingcited by 2
  8. 2026SATTC: Structure-Aware Label-Free Test-Time Calibration for Cross-Subject EEG-to-Image Retrievalcited by 2
  9. 2025Optimizing Supply Chain Networks with the Power of Graph Neural Networkscited by 1
  10. 2025Decoding Visual Neural Representations by Multimodal with Dynamic Balancingcited by 1
  11. 2026A Unified SPD Token Transformer Framework for EEG Classification: Systematic Comparison of Geometric Embeddingscited by 0
  12. 2026Toward High-Fidelity Visual Reconstruction: From EEG-Based Conditioned Generation to Joint-Modal Guided Rebuildingcited by 0