# MoRE-Brain: Routed Mixture of Experts for Interpretable and Generalizable Cross-Subject fMRI Visual Decoding

arXiv: https://arxiv.org/abs/2505.15946 · HTML: https://mlsanity.com/paper/2505.15946/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Medical and biomedical MoE, one of 12 directions in Mixture-of-experts.
- Builds on 28 of its 50 references that are in the map.
- First posted on arXiv 21 May 2025.

## Placed in

- Mixture-of-experts / [Medical and biomedical MoE](https://mlsanity.com/moe/medical-and-biomedical-moe/)

## Builds on (most cited first, 28 of 28)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data](https://mlsanity.com/paper/2306.09344/) (2023)
- [Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors](https://mlsanity.com/paper/2305.18274/) (2023)
- [MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data](https://mlsanity.com/paper/2403.11207/) (2024)
- [MindBridge: A Cross-Subject Brain Decoding Framework](https://mlsanity.com/paper/2404.07850/) (2024)
- [MixCo: Mix-up Contrastive Learning for Visual Representation](https://mlsanity.com/paper/2010.06300/) (2020)
- [MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusion](https://mlsanity.com/paper/2308.04249/) (2023)
- [Reconstructing Perceptive Images from Brain Activity by Shape-Semantic GAN](https://mlsanity.com/paper/2101.12083/) (2021)
- [Brain Diffusion for Visual Exploration: Cortical Discovery using Large Scale Generative Models](https://mlsanity.com/paper/2306.03089/) (2023)
- [Temporal Working Memory: Query-Guided Segment Refinement for Enhanced Multimodal Understanding](https://mlsanity.com/paper/2502.06020/) (2025)
- [FT2TF: First-Person Statement Text-To-Talking Face Generation](https://mlsanity.com/paper/2312.05430/) (2023)
- [Through their eyes: multi-subject Brain Decoding with simple alignment techniques](https://mlsanity.com/paper/2309.00627/) (2023)
- [The CLIP Model is Secretly an Image-to-Prompt Converter](https://mlsanity.com/paper/2305.12716/) (2023)
- [CLIP-MUSED: CLIP-Guided Multi-Subject Visual Neural Information Semantic Decoding](https://mlsanity.com/paper/2402.08994/) (2024)
- [Describe Anything in Medical Images](https://mlsanity.com/paper/2505.05804/) (2025)
- [A Deep Network for Explainable Prediction of Non-Imaging Phenotypes using Anatomical Multi-View Data](https://mlsanity.com/paper/2401.04579/) (2024)
- [Hierarchical Spatio-Temporal State-Space Modeling for fMRI Analysis](https://mlsanity.com/paper/2408.13074/) (2024)
- [BrainBits: How Much of the Brain are Generative Reconstruction Methods Using?](https://mlsanity.com/paper/2411.02783/) (2024)
- [Brain-Streams: fMRI-to-Image Reconstruction with Multi-modal Guidance](https://mlsanity.com/paper/2409.12099/) (2024)

## Built on by (most cited first, 7 of 7)

- [Prompt-based Adaptation in Large-scale Vision Models: A Survey](https://mlsanity.com/paper/2510.13219/) (2025)
- [Doctor Approved: Generating Medically Accurate Skin Disease Images through AI-Expert Feedback](https://mlsanity.com/paper/2506.12323/) (2025)
- [Music Audio-Visual Question Answering Requires Specialized Multimodal Designs](https://mlsanity.com/paper/2505.20638/) (2025)
- [FPED: A Functional-Network Prior-Guided Mixture-of-Experts Framework for Interpretable Brain Decoding](https://mlsanity.com/paper/2605.19279/) (2026)
- [HyNeuralMap: Hyperbolic Mapping of Visual Semantics to Neural Hierarchies](https://mlsanity.com/paper/2605.09392/) (2026)
- [Beyond Low-Rank: Low-Rank Sparse Prompting via Spiking Neural Network and Prompt Factorization](https://mlsanity.com/paper/2606.01945/) (2026)
- [The Platonic brain bridge hypothesis: human brain networks as an architectural prior for multimodal large language models](https://mlsanity.com/paper/2609.10947/) (2026)

## Abstract (arXiv)

Decoding visual experiences from fMRI offers a powerful avenue to understand human perception and develop advanced brain-computer interfaces. However, current progress often prioritizes maximizing reconstruction fidelity while overlooking interpretability, an essential aspect for deriving neuroscientific insight. To address this gap, we propose MoRE-Brain, a neuro-inspired framework designed for high-fidelity, adaptable, and interpretable visual reconstruction. MoRE-Brain uniquely employs a hierarchical Mixture-of-Experts architecture where distinct experts process fMRI signals from functionally related voxel groups, mimicking specialized brain networks. The experts are first trained to encode fMRI into the frozen CLIP space. A finetuned diffusion model then synthesizes images, guided by expert outputs through a novel dual-stage routing mechanism that dynamically weighs expert contributions across the diffusion process. MoRE-Brain offers three main advancements: First, it introduces a novel Mixture-of-Experts architecture grounded in brain network principles for neuro-decoding. Second, it achieves efficient cross-subject generalization by sharing core expert networks while adapting only subject-specific routers. Third, it provides enhanced mechanistic insight, as the explicit routing reveals precisely how different modeled brain regions shape the semantic and spatial attributes of the reconstructed image. Extensive experiments validate MoRE-Brain's high reconstruction fidelity, with bottleneck analyses further demonstrating its effective utilization of fMRI signals, distinguishing genuine neural decoding from over-reliance on generative priors. Consequently, MoRE-Brain marks a substantial advance towards more generalizable and interpretable fMRI-based visual decoding. Code will be publicly available soon: https://github.com/yuxiangwei0808/MoRE-Brain.