# SYNAPSE: Synergizing an Adapter and Finetuning for High-Fidelity EEG Synthesis from a CLIP-Aligned Encoder

arXiv: https://arxiv.org/abs/2511.17547 · HTML: https://mlsanity.com/paper/2511.17547/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 24 of its 35 references that are in the map.
- First posted on arXiv 11 Nov 2025.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 24 of 24)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [Adversarial Diffusion Distillation](https://mlsanity.com/paper/2311.17042/) (2023)
- [DoRA: Weight-Decomposed Low-Rank Adaptation](https://mlsanity.com/paper/2402.09353/) (2024)
- [How Do Vision Transformers Work?](https://mlsanity.com/paper/2202.06709/) (2022)
- [Deep Learning Human Mind for Automated Visual Classification](https://mlsanity.com/paper/1609.00344/) (2016)
- [DreamDiffusion: Generating High-Quality Images from Brain EEG Signals](https://mlsanity.com/paper/2306.16934/) (2023)
- [QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning](https://mlsanity.com/paper/2402.03666/) (2024)
- [TimeMAE: Self-Supervised Representations of Time Series with Decoupled Masked Autoencoders](https://mlsanity.com/paper/2303.00320/) (2023)
- [BrainVis: Exploring the Bridge between Brain and Visual Signals via Image Reconstruction](https://mlsanity.com/paper/2312.14871/) (2023)
- [Learning Robust Deep Visual Representations from EEG Brain Recordings](https://mlsanity.com/paper/2310.16532/) (2023)
- [Seeing through the Brain: Image Reconstruction of Visual Perception from Human Brain Signals](https://mlsanity.com/paper/2308.02510/) (2023)
- [Guess What I Think: Streamlined EEG-to-Image Generation with Latent Diffusion Models](https://mlsanity.com/paper/2410.02780/) (2024)
- [Natural Image Reconstruction from fMRI using Deep Learning: A Survey](https://mlsanity.com/paper/2110.09006/) (2021)
- [SDSC:A Structure-Aware Metric for Semantic Signal Representation Learning](https://mlsanity.com/paper/2507.14516/) (2025)

## Built on by (most cited first, 1 of 1)

- [Toward High-Fidelity Visual Reconstruction: From EEG-Based Conditioned Generation to Joint-Modal Guided Rebuilding](https://mlsanity.com/paper/2603.19667/) (2026)

## Abstract (arXiv)

Recent progress in diffusion-based generative models has enabled high-quality image synthesis conditioned on diverse modalities. Extending such models to brain signals could deepen our understanding of human perception and mental representations. However,electroencephalography (EEG) presents major challenges for image generation due to high noise, low spatial resolution, and strong inter-subject variability. Existing approaches,such as DreamDiffusion, BrainVis, and GWIT, primarily adapt EEG features to pre-trained Stable Diffusion models using complex alignment or classification pipelines, often resulting in large parameter counts and limited interpretability. We introduce SYNAPSE, a two-stage framework that bridges EEG signal representation learning and high-fidelity image synthesis. In Stage1, a CLIP-aligned EEG autoencoder learns a semantically structured latent representation by combining signal reconstruction and cross-modal alignment objectives. In Stage2, the pretrained encoder is frozen and integrated with a lightweight adaptation of Stable Diffusion, enabling efficient conditioning on EEG features with minimal trainable parameters. Our method achieves a semantically coherent latent space and state-of-the-art perceptual fidelity on the CVPR40 dataset, outperforming prior EEG-to-image models in both reconstruction efficiency and image quality. Quantitative and qualitative analyses demonstrate that SYNAPSE generalizes effectively across subjects, preserving visual semantics even when class-level agreement is reduced. These results suggest that reconstructing what the brain perceives, rather than what it classifies, is key to faithful EEG-based image generation.