# Perception Activator: An intuitive and portable framework for brain cognitive exploration

arXiv: https://arxiv.org/abs/2507.02311 · HTML: https://mlsanity.com/paper/2507.02311/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 34 references that are in the map.
- First posted on arXiv 3 Jul 2025.

## Builds on (most cited first, 19 of 19)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors](https://mlsanity.com/paper/2305.18274/) (2023)
- [Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decoding](https://mlsanity.com/paper/2211.06956/) (2022)
- [Generic decoding of seen and imagined objects using hierarchical visual features](https://mlsanity.com/paper/1510.06479/) (2015)
- [Natural scene reconstruction from fMRI signals using generative latent diffusion](https://mlsanity.com/paper/2303.05334/) (2023)
- [Mind Reader: Reconstructing complex images from brain activities](https://mlsanity.com/paper/2210.01769/) (2022)
- [Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Features](https://mlsanity.com/paper/2210.06756/) (2022)
- [MixCo: Mix-up Contrastive Learning for Visual Representation](https://mlsanity.com/paper/2010.06300/) (2020)
- [Decoding Brain Representations by Multimodal Learning of Neural Activity and Visual Features](https://mlsanity.com/paper/1810.10974/) (2018)
- [MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusion](https://mlsanity.com/paper/2308.04249/) (2023)
- [UniBrain: Unify Image Reconstruction and Captioning All in One Diffusion Model from Human Brain Activity](https://mlsanity.com/paper/2308.07428/) (2023)
- [BrainCLIP: Bridging Brain and Visual-Linguistic Representation Via CLIP for Generic Natural Visual Stimulus Decoding](https://mlsanity.com/paper/2302.12971/) (2023)
- [MindTuner: Cross-Subject Visual Decoding with Visual Fingerprint and Semantic Correction](https://mlsanity.com/paper/2404.12630/) (2024)
- [NeuroCLIP: Neuromorphic Data Understanding by CLIP and SNN](https://mlsanity.com/paper/2306.12073/) (2023)
- [Wills Aligner: Multi-Subject Collaborative Brain Visual Decoding](https://mlsanity.com/paper/2404.13282/) (2024)

## Abstract (arXiv)

Recent advances in brain-vision decoding have driven significant progress, reconstructing with high fidelity perceived visual stimuli from neural activity, e.g., functional magnetic resonance imaging (fMRI), in the human visual cortex. Most existing methods decode the brain signal using a two-level strategy, i.e., pixel-level and semantic-level. However, these methods rely heavily on low-level pixel alignment yet lack sufficient and fine-grained semantic alignment, resulting in obvious reconstruction distortions of multiple semantic objects. To better understand the brain's visual perception patterns and how current decoding models process semantic objects, we have developed an experimental framework that uses fMRI representations as intervention conditions. By injecting these representations into multi-scale image features via cross-attention, we compare both downstream performance and intermediate feature changes on object detection and instance segmentation tasks with and without fMRI information. Our results demonstrate that incorporating fMRI signals enhances the accuracy of downstream detection and segmentation, confirming that fMRI contains rich multi-object semantic cues and coarse spatial localization information-elements that current models have yet to fully exploit or integrate.