# MindCine: Multimodal EEG-to-Video Reconstruction with Large-Scale Pretrained Models

arXiv: https://arxiv.org/abs/2601.18192 · HTML: https://mlsanity.com/paper/2601.18192/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 22 references that are in the map.
- First posted on arXiv 26 Jan 2026.

## Builds on (most cited first, 13 of 13)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI](https://mlsanity.com/paper/2405.18765/) (2024)
- [CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding](https://mlsanity.com/paper/2412.07236/) (2024)
- [Cinematic Mindscapes: High-quality Video Reconstruction from Brain Activity](https://mlsanity.com/paper/2305.11675/) (2023)
- [SketchyCOCO: Image Generation from Freehand Scene Sketches](https://mlsanity.com/paper/2003.02683/) (2020)
- [SoftCLIP: Softer Cross-modal Alignment Makes CLIP Stronger](https://mlsanity.com/paper/2303.17561/) (2023)
- [NeuroClips: Towards High-fidelity and Smooth fMRI-to-Video Reconstruction](https://mlsanity.com/paper/2410.19452/) (2024)
- [Animate Your Thoughts: Decoupled Reconstruction of Dynamic Natural Vision from Slow Brain Activity](https://mlsanity.com/paper/2405.03280/) (2024)
- [CineBrain: A Large-Scale Multi-Modal Brain Dataset During Naturalistic Audiovisual Narrative Processing](https://mlsanity.com/paper/2503.06940/) (2025)

## Built on by (most cited first, 1 of 1)

- [Replaying the Movie in Your Mind Decoding Dynamic Visual Perception and Recall from EEG](https://mlsanity.com/paper/2610.13120/) (2026)

## Abstract (arXiv)

Reconstructing human dynamic visual perception from electroencephalography (EEG) signals is of great research significance since EEG's non-invasiveness and high temporal resolution. However, EEG-to-video reconstruction remains challenging due to: 1) Single Modality: existing studies solely align EEG signals with the text modality, which ignores other modalities and are prone to suffer from overfitting problems; 2) Data Scarcity: current methods often have difficulty training to converge with limited EEG-video data. To solve the above problems, we propose a novel framework MindCine to achieve high-fidelity video reconstructions on limited data. We employ a multimodal joint learning strategy to incorporate beyond-text modalities in the training stage and leverage a pre-trained large EEG model to relieve the data scarcity issue for decoding semantic information, while a Seq2Seq model with causal attention is specifically designed for decoding perceptual information. Extensive experiments demonstrate that our model outperforms state-of-the-art methods both qualitatively and quantitatively. Additionally, the results underscore the effectiveness of the complementary strengths of different modalities and demonstrate that leveraging a large-scale EEG model can further enhance reconstruction performance by alleviating the challenges associated with limited data.