mlsanityData to 7 Oct 2026

DreamDiffusion: Generating High-Quality Images from Brain EEG Signals

arXiv 2306.16934 ↗First posted 29 Jun 2023cs.CVcited by 56

Not on the map. Why?

Abstract (arXiv)

This paper introduces DreamDiffusion, a novel method for generating high-quality images directly from brain electroencephalogram (EEG) signals, without the need to translate thoughts into text. DreamDiffusion leverages pre-trained text-to-image models and employs temporal masked signal modeling to pre-train the EEG encoder for effective and robust EEG representations. Additionally, the method further leverages the CLIP image encoder to provide extra supervision to better align EEG, text, and image embeddings with limited EEG-image pairs. Overall, the proposed method overcomes the challenges of using EEG signals for image generation, such as noise, limited information, and individual differences, and achieves promising results. Quantitative and qualitative results demonstrate the effectiveness of the proposed method as a significant step towards portable and low-cost ``thoughts-to-image'', with potential applications in neuroscience and computer vision. The code is available here urlhttps://github.com/bbaaii/DreamDiffusion.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Builds on 32 of its 45 references that are in the map. First posted on arXiv 29 Jun 2023.

Builds on

32 of 45 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  6. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  7. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  8. 2020Denoising Diffusion Implicit Modelscited by 9,655
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  10. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  11. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  12. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
20 more
  1. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2022Generative Adversarial Networkscited by 5,603
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  6. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  7. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  8. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  9. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  10. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  11. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  12. 2021Score-based Generative Modeling in Latent Spacecited by 634
  13. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
  14. 2021On Fast Sampling of Diffusion Probabilistic Modelscited by 170
  15. 2022Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decodingcited by 142
  16. 2021Noise Estimation for Generative Diffusion Modelscited by 94
  17. 2018MOABB: Trustworthy algorithm benchmarking for BCIscited by 80
  18. 2022Reconstruction of Perceived Images from fMRI Patterns and Semantic Brain Exploration using Instance-Conditioned GANscited by 55
  19. 2020Reconstructing Natural Scenes from fMRI Patterns using BigBiGANcited by 44
  20. 2023TextIR: A Simple Framework for Text-based Editable Image Restorationcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 22; 2025: 18; 2026: 9.

56 papers.

Sort:

By area:

  1. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  2. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 82
  3. 2023BrainVis: Exploring the Bridge between Brain and Visual Signals via Image Reconstructioncited by 30
  4. 2023ViT-Lens: Towards Omni-modal Representationscited by 29
  5. 2025Bridging the Vision-Brain Gap with an Uncertainty-Aware Blur Priorcited by 28
  6. 2024Neuro-3D: Towards 3D Visual Decoding from EEG Signalscited by 23
  7. 2024Visual Neural Decoding via Improved Visual-EEG Semantic Consistencycited by 21
  8. 2024ClassDiffusion: More Aligned Personalization Tuning with Explicit Class Guidancecited by 18
  9. 2024Thought2Text: Text Generation from EEG Signal using Large Language Models (LLMs)cited by 18
  10. 2024Generative AI Enables EEG Super-Resolution via Spatio-Temporal Adaptive Diffusion Learningcited by 17
  11. 2023Seeing through the Brain: Image Reconstruction of Visual Perception from Human Brain Signalscited by 16
  12. 2023Brain-Conditional Multimodal Synthesis: A Survey and Taxonomycited by 15
38 more
  1. 2024Enhancing EEG-to-Text Decoding through Transferable Representations from Pre-trained Contrastive EEG-Text Masked Autoencodercited by 15
  2. 2024Guess What I Think: Streamlined EEG-to-Image Generation with Latent Diffusion Modelscited by 11
  3. 2025CineBrain: A Large-Scale Multi-Modal Brain Dataset During Naturalistic Audiovisual Narrative Processingcited by 11
  4. 2023Decoding visual brain representations from electroencephalography through Knowledge Distillation and latent diffusion modelscited by 10
  5. 2024NeuSpeech: Decode Neural signal as Speechcited by 10
  6. 2024NECOMIMI: Neural-Cognitive Multimodal EEG-informed Image Generation with Diffusion Modelscited by 10
  7. 2024TMPQ-DM: Joint Timestep Reduction and Quantization Precision Selection for Efficient Diffusion Modelscited by 9
  8. 2024Towards Neural Foundation Models for Vision: Aligning EEG, MEG, and fMRI Representations for Decoding, Encoding, and Modality Conversioncited by 9
  9. 2024Naturalistic Music Decoding from EEG Data via Latent Diffusion Modelscited by 8
  10. 2026Let EEG Models Learn EEGcited by 6
  11. 2024You Only Acquire Sparse-channel (YOAS): A Unified Framework for Dense-channel EEG Generationcited by 5
  12. 2024Multi-modal Mood Reader: Pre-trained Model Empowers Cross-Subject Emotion Recognitioncited by 4
  13. 2024BrainDreamer: Reasoning-Coherent and Controllable Image Generation from EEG Brain Signals via Language Guidancecited by 4
  14. 2025Foundation Models for Cross-Domain EEG Analysis Application: A Surveycited by 4
  15. 2024A Survey of Spatio-Temporal EEG data Analysis: from Models to Applicationscited by 3
  16. 2024EEG-Driven 3D Object Reconstruction with Style Consistency and Diffusion Priorcited by 3
  17. 2025A Survey on Bridging EEG Signals and Generative AI: From Image and Text to Beyondcited by 3
  18. 2025ViEEG: Hierarchical Visual Neural Representation for EEG Brain Decodingcited by 3
  19. 2025MoTime: A Dataset Suite for Multimodal Time Series Forecastingcited by 3
  20. 2025Category-aware EEG image generation based on wavelet transform and contrast semantic losscited by 3
  21. 2024Alljoined1 -- A dataset for EEG-to-Image decodingcited by 2
  22. 2025Integrating Biological and Machine Intelligence: Attention Mechanisms in Brain-Computer Interfacescited by 2
  23. 2025EgoBrain: Synergizing Minds and Eyes For Human Action Understandingcited by 2
  24. 2026Learning Brain Representation with Hierarchical Visual Embeddingscited by 2
  25. 2026Subject-Aware Multi-Granularity Alignment for Zero-Shot EEG-to-Image Retrievalcited by 2
  26. 2024BrainDecoder: Style-Based Visual Decoding of EEG Signalscited by 1
  27. 2025Self-Calibrating BCIs: Ranking and Recovery of Mental Targets Without Labelscited by 1
  28. 20253D-Telepathy: Reconstructing 3D Objects from EEG Signalscited by 1
  29. 2025Decoding Visual Neural Representations by Multimodal with Dynamic Balancingcited by 1
  30. 2025Unveiling Deep Semantic Uncertainty Perception for Language-Anchored Multi-modal Vision-Brain Alignmentcited by 1
  31. 2025SYNAPSE: Synergizing an Adapter and Finetuning for High-Fidelity EEG Synthesis from a CLIP-Aligned Encodercited by 1
  32. 2026EEG2Vision: A Multimodal EEG-Based Framework for 2D Visual Reconstruction in Cognitive Neurosciencecited by 1
  33. 2023Mapping EEG Signals to Visual Stimuli: A Deep Learning Approach to Match vs. Mismatch Classificationcited by 0
  34. 2023On the recognition of the game type based on physiological signals and eye trackingcited by 0
  35. 2024Dynamic Neural Communication: Convergence of Computer Vision and Brain-Computer Interfacecited by 0
  36. 2025CATVis: Context-Aware Thought Visualizationcited by 0
  37. 2025BRAIN: Bias-Mitigation Continual Learning Approach to Vision-Brain Understandingcited by 0
  38. 2025Uni-Neur2Img: Unified Neural Signal-Guided Image Generation, Editing, and Stylization via Diffusion Transformerscited by 0

50 most cited of 56 All 56 (JSON)