# Bridging the Vision-Brain Gap with an Uncertainty-Aware Blur Prior

arXiv: https://arxiv.org/abs/2503.04207 · HTML: https://mlsanity.com/paper/2503.04207/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 83 references that are in the map.
- First posted on arXiv 6 Mar 2025.

## Builds on (most cited first, 30 of 43)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [MedCLIP: Contrastive Learning from Unpaired Medical Images and Text](https://mlsanity.com/paper/2210.10163/) (2022)
- [Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning](https://mlsanity.com/paper/2203.02053/) (2022)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [AudioCLIP: Extending CLIP to Image, Text and Audio](https://mlsanity.com/paper/2106.13043/) (2021)
- [In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06329/) (2021)
- [OpenOOD: Benchmarking Generalized Out-of-Distribution Detection](https://mlsanity.com/paper/2210.07242/) (2022)
- [ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding](https://mlsanity.com/paper/2212.05171/) (2022)
- [A Survey on Deep Semi-supervised Learning](https://mlsanity.com/paper/2103.00550/) (2021)
- [Trusted Multi-View Classification with Dynamic Evidential Fusion](https://mlsanity.com/paper/2204.11423/) (2022)
- [Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts](https://mlsanity.com/paper/2006.09239/) (2020)
- [BioCLIP: A Vision Foundation Model for the Tree of Life](https://mlsanity.com/paper/2311.18803/) (2023)
- [Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors](https://mlsanity.com/paper/2305.18274/) (2023)
- [Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decoding](https://mlsanity.com/paper/2211.06956/) (2022)
- [MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data](https://mlsanity.com/paper/2403.11207/) (2024)
- [Uncertainty Aware Semi-Supervised Learning on Graph Data](https://mlsanity.com/paper/2010.12783/) (2020)
- [Provable Dynamic Fusion for Low-Quality Multimodal Data](https://mlsanity.com/paper/2306.02050/) (2023)
- [Decoding Natural Images from EEG for Object Recognition](https://mlsanity.com/paper/2308.13234/) (2023)
- [Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion](https://mlsanity.com/paper/2403.07721/) (2024)
- [Learning Audio-Video Modalities from Image Captions](https://mlsanity.com/paper/2204.00679/) (2022)
- [From voxels to pixels and back: Self-supervision in natural-image reconstruction from fMRI](https://mlsanity.com/paper/1907.02431/) (2019)
- [Brain decoding: toward real-time reconstruction of visual perception](https://mlsanity.com/paper/2310.19812/) (2023)
- [MindBridge: A Cross-Subject Brain Decoding Framework](https://mlsanity.com/paper/2404.07850/) (2024)
- [Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Features](https://mlsanity.com/paper/2210.06756/) (2022)

## Built on by (most cited first, 29 of 29)

- [NeuroBridge: Bio-Inspired Self-Supervised EEG-to-Image Decoding via Cognitive Priors and Bidirectional Semantic Alignment](https://mlsanity.com/paper/2511.06836/) (2025)
- [Aligning What EEG Can See: Structural Representations for Brain-Vision Matching](https://mlsanity.com/paper/2603.07077/) (2026)
- [HyFI: Hyperbolic Feature Interpolation for Brain-Vision Alignment](https://mlsanity.com/paper/2603.22721/) (2026)
- [Deep Models, Shallow Alignment: Uncovering the Granularity Mismatch in Neural Decoding](https://mlsanity.com/paper/2601.21948/) (2026)
- [Subject-Aware Multi-Granularity Alignment for Zero-Shot EEG-to-Image Retrieval](https://mlsanity.com/paper/2604.17782/) (2026)
- [Spatial-Functional awareness Transformer-based graph archetype contrastive learning for Decoding Visual Neural Representations from EEG](https://mlsanity.com/paper/2509.24761/) (2025)
- [Learning Brain Representation with Hierarchical Visual Embeddings](https://mlsanity.com/paper/2602.07495/) (2026)
- [CognitionCapturerPro: Towards High-Fidelity Visual Decoding from EEG/MEG via Multi-modal Information and Asymmetric Alignment](https://mlsanity.com/paper/2603.12722/) (2026)
- [Unveiling Deep Semantic Uncertainty Perception for Language-Anchored Multi-modal Vision-Brain Alignment](https://mlsanity.com/paper/2511.04078/) (2025)
- [Multi-Level Bidirectional Biomimetic Learning for EEG-Based Visual Decoding](https://mlsanity.com/paper/2605.04680/) (2026)
- [Rethinking Brain Decoding with CLIP: The Role of Adversarial Robustness](https://mlsanity.com/paper/2607.03165/) (2026)
- [SCORE: Subject Coordinate Recovery for Label-Free Cross-Subject EEG-to-Image Retrieval](https://mlsanity.com/paper/2608.19134/) (2026)
- [BRAIN: Bias-Mitigation Continual Learning Approach to Vision-Brain Understanding](https://mlsanity.com/paper/2508.18187/) (2025)
- [Brain-Inspired Capture: Evidence-Driven Neuromimetic Perceptual Simulation for Visual Decoding](https://mlsanity.com/paper/2604.17927/) (2026)
- [SIMON: Saliency-aware Integrative Multi-view Object-centric Neural Decoding](https://mlsanity.com/paper/2605.00401/) (2026)
- [StableMind: Source-Free Cross-Subject fMRI Decoding with Regularized Adaptation](https://mlsanity.com/paper/2605.02586/) (2026)
- [Neural Visual Decoding via Cognitive guided Adaptive Blurring and Information Constrained Alignment](https://mlsanity.com/paper/2605.16418/) (2026)
- [MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding](https://mlsanity.com/paper/2605.24523/) (2026)
- [SUP-MCRL: Subject-aware Unified Pseudo-feature Coded Multimodal Contrastive Representation Learning for EEG Visual Decoding](https://mlsanity.com/paper/2606.16615/) (2026)
- [What Does the Brain See? Multiview Neural Representations to Demystify the Brain-Visual Alignment](https://mlsanity.com/paper/2606.25718/) (2026)
- [FSDBN: Foreground-Aware EEG-Visual Alignment via Dynamic Brain Networks](https://mlsanity.com/paper/2607.18344/) (2026)
- [EEG-EditBench: Probing Visual Information in EEG-Image Retrieval Models with Controlled Image Edits](https://mlsanity.com/paper/2607.27857/) (2026)
- [Beyond Trial Averaging: Anchoring Neural and Visual Representations for Few-Repetition Brain-to-Image Retrieval](https://mlsanity.com/paper/2608.19128/) (2026)
- [EEG-VID: Task-Guided Latent Predictive Pretraining for EEG Decoding and Assistive Target Selection](https://mlsanity.com/paper/2609.00566/) (2026)
- [Brain-to-Image Generation: Reconstructing Visual Stimuli from EEG using Generative Adversarial Networks](https://mlsanity.com/paper/2609.22282/) (2026)
- [Adaptive Cortically Constrained EEG-Vision Alignment for Zero-Shot Brain-to-Image Retrieval](https://mlsanity.com/paper/2609.24109/) (2026)
- [The Visual Target Matters: Learning across the Visual Hierarchy for Brain-to-Image Retrieval](https://mlsanity.com/paper/2609.24136/) (2026)
- [RPA: Residual Patch-Token Adapter for Image Retrieval from EEG and MEG](https://mlsanity.com/paper/2609.31698/) (2026)
- [Structured Visual Target Learning For Cross-Subject eeg-to-image retrieval](https://mlsanity.com/paper/2609.36971/) (2026)

## Abstract (arXiv)

Can our brain signals faithfully reflect the original visual stimuli, even including high-frequency details? Although human perceptual and cognitive capacities enable us to process and remember visual information, these abilities are constrained by several factors, such as limited attentional resources and the finite capacity of visual memory. When visual stimuli are processed by human visual system into brain signals, some information is inevitably lost, leading to a discrepancy known as the System GAP. Additionally, perceptual and cognitive dynamics, along with technical noise in signal acquisition, degrade the fidelity of brain signals relative to the visual stimuli, known as the Random GAP. When encoded brain representations are directly aligned with the corresponding pretrained image features, the System GAP and Random GAP between paired data challenge the model, requiring it to bridge these gaps. However, in the context of limited paired data, these gaps are difficult for the model to learn, leading to overfitting and poor generalization to new data. To address these GAPs, we propose a simple yet effective approach called the Uncertainty-aware Blur Prior (UBP). It estimates the uncertainty within the paired data, reflecting the mismatch between brain signals and visual stimuli. Based on this uncertainty, UBP dynamically blurs the high-frequency details of the original images, reducing the impact of the mismatch and improving alignment. Our method achieves a top-1 accuracy of 50.9% and a top-5 accuracy of 79.7% on the zero-shot brain-to-image retrieval task, surpassing previous state-of-the-art methods by margins of 13.7% and 9.8%, respectively. Code is available at hrefhttps://github.com/HaitaoWuTJU/Uncertainty-aware-Blur-PriorGitHub.