# Neuro-3D: Towards 3D Visual Decoding from EEG Signals

arXiv: https://arxiv.org/abs/2411.12248 · HTML: https://mlsanity.com/paper/2411.12248/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 82 references that are in the map.
- First posted on arXiv 19 Nov 2024.

## Builds on (most cited first, 30 of 46)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [Point-E: A System for Generating 3D Point Clouds from Complex Prompts](https://mlsanity.com/paper/2212.08751/) (2022)
- [Wonder3D: Single Image to 3D using Cross-Domain Diffusion](https://mlsanity.com/paper/2310.15008/) (2023)
- [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021)
- [LION: Latent Point Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2210.06978/) (2022)
- [3D Shape Generation and Completion through Point-Voxel Diffusion](https://mlsanity.com/paper/2104.03670/) (2021)
- [Point-Voxel CNN for Efficient 3D Deep Learning](https://mlsanity.com/paper/1907.03739/) (2019)
- [PointLLM: Empowering Large Language Models to Understand Point Clouds](https://mlsanity.com/paper/2308.16911/) (2023)
- [Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI](https://mlsanity.com/paper/2405.18765/) (2024)
- [Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors](https://mlsanity.com/paper/2305.18274/) (2023)
- [Generic decoding of seen and imagined objects using hierarchical visual features](https://mlsanity.com/paper/1510.06479/) (2015)
- [MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data](https://mlsanity.com/paper/2403.11207/) (2024)
- [Mind Reader: Reconstructing complex images from brain activities](https://mlsanity.com/paper/2210.01769/) (2022)
- [PC2: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstruction](https://mlsanity.com/paper/2302.10668/) (2023)
- [Decoding Natural Images from EEG for Object Recognition](https://mlsanity.com/paper/2308.13234/) (2023)

## Built on by (most cited first, 23 of 23)

- [CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding](https://mlsanity.com/paper/2506.23075/) (2025)
- [CineBrain: A Large-Scale Multi-Modal Brain Dataset During Naturalistic Audiovisual Narrative Processing](https://mlsanity.com/paper/2503.06940/) (2025)
- [SynBrain: Enhancing Visual-to-fMRI Synthesis via Probabilistic Representation Learning](https://mlsanity.com/paper/2508.10298/) (2025)
- [Meta-Learning an In-Context Transformer Model of Human Higher Visual Cortex](https://mlsanity.com/paper/2505.15813/) (2025)
- [Foundation Models for Cross-Domain EEG Analysis Application: A Survey](https://mlsanity.com/paper/2508.15716/) (2025)
- [Multigranular Evaluation for Brain Visual Decoding](https://mlsanity.com/paper/2507.07993/) (2025)
- [3D-Telepathy: Reconstructing 3D Objects from EEG Signals](https://mlsanity.com/paper/2506.21843/) (2025)
- [EEG2Vision: A Multimodal EEG-Based Framework for 2D Visual Reconstruction in Cognitive Neuroscience](https://mlsanity.com/paper/2604.08063/) (2026)
- [Meta-learning In-Context Enables Training-Free Cross Subject Brain Decoding](https://mlsanity.com/paper/2604.08537/) (2026)
- [STAMBRIDGE: Spectral-Temporal Amplitude-aware Mid-Feature Bridge for EEG Visual Decoding](https://mlsanity.com/paper/2605.23137/) (2026)
- [TAlignDiff: Automatic Tooth Alignment assisted by Diffusion-based Transformation Learning](https://mlsanity.com/paper/2508.04565/) (2025)
- [MindPilot: Closed-loop Visual Stimulation Optimization for Brain Modulation with EEG-guided Diffusion](https://mlsanity.com/paper/2602.10552/) (2026)
- [EEG-Based Brain-LLM Interface for Human Preference Aligned Generation](https://mlsanity.com/paper/2603.16897/) (2026)
- [Brain3D: EEG-to-3D Decoding of Visual Representations via Multimodal Reasoning](https://mlsanity.com/paper/2604.08068/) (2026)
- [NeuroFlow: Toward Unified Visual Encoding and Decoding from Neural Activity](https://mlsanity.com/paper/2604.09817/) (2026)
- [MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding](https://mlsanity.com/paper/2605.24523/) (2026)
- [SUP-MCRL: Subject-aware Unified Pseudo-feature Coded Multimodal Contrastive Representation Learning for EEG Visual Decoding](https://mlsanity.com/paper/2606.16615/) (2026)
- [Dual-Stream EEG Decoding for 3D Visual Perception](https://mlsanity.com/paper/2606.22182/) (2026)
- [FSDBN: Foreground-Aware EEG-Visual Alignment via Dynamic Brain Networks](https://mlsanity.com/paper/2607.18344/) (2026)
- [EEG-VID: Task-Guided Latent Predictive Pretraining for EEG Decoding and Assistive Target Selection](https://mlsanity.com/paper/2609.00566/) (2026)
- [Mind2Cloud: EEG-to-Point Cloud Generation with Two-Granularity Diffusion Decoding](https://mlsanity.com/paper/2609.13991/) (2026)
- [Adaptive Cortically Constrained EEG-Vision Alignment for Zero-Shot Brain-to-Image Retrieval](https://mlsanity.com/paper/2609.24109/) (2026)
- [The Visual Target Matters: Learning across the Visual Hierarchy for Brain-to-Image Retrieval](https://mlsanity.com/paper/2609.24136/) (2026)

## Abstract (arXiv)

Human's perception of the visual world is shaped by the stereo processing of 3D information. Understanding how the brain perceives and processes 3D visual stimuli in the real world has been a longstanding endeavor in neuroscience. Towards this goal, we introduce a new neuroscience task: decoding 3D visual perception from EEG signals, a neuroimaging technique that enables real-time monitoring of neural dynamics enriched with complex visual cues. To provide the essential benchmark, we first present EEG-3D, a pioneering dataset featuring multimodal analysis data and extensive EEG recordings from 12 subjects viewing 72 categories of 3D objects rendered in both videos and images. Furthermore, we propose Neuro-3D, a 3D visual decoding framework based on EEG signals. This framework adaptively integrates EEG features derived from static and dynamic stimuli to learn complementary and robust neural representations, which are subsequently utilized to recover both the shape and color of 3D objects through the proposed diffusion-based colored point cloud decoder. To the best of our knowledge, we are the first to explore EEG-based 3D visual decoding. Experiments indicate that Neuro-3D not only reconstructs colored 3D objects with high fidelity, but also learns effective neural representations that enable insightful brain region analysis. The dataset and associated code will be made publicly available.