# Decoding visual brain representations from electroencephalography through Knowledge Distillation and latent diffusion models

arXiv: https://arxiv.org/abs/2309.07149 · HTML: https://mlsanity.com/paper/2309.07149/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 11 of its 30 references that are in the map.
- First posted on arXiv 8 Sep 2023.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 11 of 11)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Learnable latent embeddings for joint behavioral and neural analysis](https://mlsanity.com/paper/2204.00673/) (2022)
- [Deep Learning Human Mind for Automated Visual Classification](https://mlsanity.com/paper/1609.00344/) (2016)
- [Measuring the Quality of Explanations: The System Causability Scale (SCS). Comparing Human and Machine Explanations](https://mlsanity.com/paper/1912.09024/) (2019)
- [DreamDiffusion: Generating High-Quality Images from Brain EEG Signals](https://mlsanity.com/paper/2306.16934/) (2023)
- [Reconstruction of Perceived Images from fMRI Patterns and Semantic Brain Exploration using Instance-Conditioned GANs](https://mlsanity.com/paper/2202.12692/) (2022)
- [Brain Captioning: Decoding human brain activity into images and text](https://mlsanity.com/paper/2305.11560/) (2023)
- [EEG2IMAGE: Image Reconstruction from EEG Brain Signals](https://mlsanity.com/paper/2302.10121/) (2023)
- [Training on the test set? An analysis of Spampinato et al. [31]](https://mlsanity.com/paper/1812.07697/) (2018)

## Built on by (most cited first, 10 of 10)

- [Guess What I Think: Streamlined EEG-to-Image Generation with Latent Diffusion Models](https://mlsanity.com/paper/2410.02780/) (2024)
- [Towards Neural Foundation Models for Vision: Aligning EEG, MEG, and fMRI Representations for Decoding, Encoding, and Modality Conversion](https://mlsanity.com/paper/2411.09723/) (2024)
- [Human-Aligned Image Models Improve Visual Decoding from the Brain](https://mlsanity.com/paper/2502.03081/) (2025)
- [EEG-Driven 3D Object Reconstruction with Style Consistency and Diffusion Prior](https://mlsanity.com/paper/2410.20981/) (2024)
- [Comprehensive Review of EEG-to-Output Research: Decoding Neural Signals into Images, Videos, and Audio](https://mlsanity.com/paper/2412.19999/) (2024)
- [EEG-VLM: A Hierarchical Vision-Language Model with Multi-Level Feature Alignment and Visually Enhanced Language-Guided Reasoning for EEG Image-Based Sleep Stage Prediction](https://mlsanity.com/paper/2511.19155/) (2025)
- [Optimized two-stage AI-based Neural Decoding for Enhanced Visual Stimulus Reconstruction from fMRI Data](https://mlsanity.com/paper/2412.13237/) (2024)
- [Neuroscience-inspired Staged Representation Learning with Disentangled Coarse- and Fine-Grained Semantics for EEG Visual Decoding](https://mlsanity.com/paper/2605.16923/) (2026)
- [What Does the Brain See? Multiview Neural Representations to Demystify the Brain-Visual Alignment](https://mlsanity.com/paper/2606.25718/) (2026)
- [Visual Semantic Decoding of Electrocorticography from Video Stimuli using End-to-End Deep Learning](https://mlsanity.com/paper/2607.18923/) (2026)

## Abstract (arXiv)

Decoding visual representations from human brain activity has emerged as a thriving research domain, particularly in the context of brain-computer interfaces. Our study presents an innovative method that employs to classify and reconstruct images from the ImageNet dataset using electroencephalography (EEG) data from subjects that had viewed the images themselves (i.e. "brain decoding"). We analyzed EEG recordings from 6 participants, each exposed to 50 images spanning 40 unique semantic categories. These EEG readings were converted into spectrograms, which were then used to train a convolutional neural network (CNN), integrated with a knowledge distillation procedure based on a pre-trained Contrastive Language-Image Pre-Training (CLIP)-based image classification teacher network. This strategy allowed our model to attain a top-5 accuracy of 80%, significantly outperforming a standard CNN and various RNN-based benchmarks. Additionally, we incorporated an image reconstruction mechanism based on pre-trained latent diffusion models, which allowed us to generate an estimate of the images which had elicited EEG activity. Therefore, our architecture not only decodes images from neural activity but also offers a credible image reconstruction from EEG only, paving the way for e.g. swift, individualized feedback experiments. Our research represents a significant step forward in connecting neural signals with visual cognition.