# Learning shared neural manifolds from multi-subject FMRI data

arXiv: https://arxiv.org/abs/2201.00622 · HTML: https://mlsanity.com/paper/2201.00622/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 45 references that are in the map.
- First posted on arXiv 22 Dec 2021.

## Builds on (most cited first, 11 of 11)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Topological Autoencoders](https://mlsanity.com/paper/1906.00722/) (2019)
- [Uncovering the Topology of Time-Varying fMRI Data using Cubical Persistence](https://mlsanity.com/paper/2006.07882/) (2020)
- [Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity](https://mlsanity.com/paper/2107.04084/) (2021)
- [Extendable and invertible manifold learning with geometry regularized autoencoders](https://mlsanity.com/paper/2007.07142/) (2020)
- [Diffusion Nets](https://mlsanity.com/paper/1506.07840/) (2015)
- [Visualizing the PHATE of Neural Networks](https://mlsanity.com/paper/1908.02831/) (2019)
- [Coarse Graining of Data via Inhomogeneous Diffusion Condensation](https://mlsanity.com/paper/1907.04463/) (2019)
- [Mapping Between fMRI Responses to Movies and their Natural Language Annotations](https://mlsanity.com/paper/1610.03914/) (2016)
- [A Convolutional Autoencoder for Multi-Subject fMRI Data Aggregation](https://mlsanity.com/paper/1608.04846/) (2016)
- [Mapping individual differences in cortical architecture using multi-view representation learning](https://mlsanity.com/paper/2004.02804/) (2020)

## Built on by (most cited first, 8 of 8)

- [MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data](https://mlsanity.com/paper/2403.11207/) (2024)
- [MindTuner: Cross-Subject Visual Decoding with Visual Fingerprint and Semantic Correction](https://mlsanity.com/paper/2404.12630/) (2024)
- [The Wisdom of a Crowd of Brains: A Universal Brain Encoder](https://mlsanity.com/paper/2406.12179/) (2024)
- [Geometry-Aware Generative Autoencoders for Warped Riemannian Metric Learning and Generative Modeling on Data Manifolds](https://mlsanity.com/paper/2410.12779/) (2024)
- [Domain Adaptation and Generalization on Functional Medical Images: A Systematic Survey](https://mlsanity.com/paper/2212.03176/) (2022)
- ["Task-relevant autoencoding" enhances machine learning for human neuroscience](https://mlsanity.com/paper/2208.08478/) (2022)
- [Mapping minds not averages: a scalable subject-specific manifold learning framework for neuroimaging data](https://mlsanity.com/paper/2505.00196/) (2025)
- [Revealing Neurocognitive and Behavioral Patterns by Unsupervised Manifold Learning from Dynamic Brain Data](https://mlsanity.com/paper/2508.11672/) (2025)

## Abstract (arXiv)

Functional magnetic resonance imaging (fMRI) is a notoriously noisy measurement of brain activity because of the large variations between individuals, signals marred by environmental differences during collection, and spatiotemporal averaging required by the measurement resolution. In addition, the data is extremely high dimensional, with the space of the activity typically having much lower intrinsic dimension. In order to understand the connection between stimuli of interest and brain activity, and analyze differences and commonalities between subjects, it becomes important to learn a meaningful embedding of the data that denoises, and reveals its intrinsic structure. Specifically, we assume that while noise varies significantly between individuals, true responses to stimuli will share common, low-dimensional features between subjects which are jointly discoverable. Similar approaches have been exploited previously but they have mainly used linear methods such as PCA and shared response modeling (SRM). In contrast, we propose a neural network called MRMD-AE (manifold-regularized multiple decoder, autoencoder), that learns a common embedding from multiple subjects in an experiment while retaining the ability to decode to individual raw fMRI signals. We show that our learned common space represents an extensible manifold (where new points not seen during training can be mapped), improves the classification accuracy of stimulus features of unseen timepoints, as well as improves cross-subject translation of fMRI signals. We believe this framework can be used for many downstream applications such as guided brain-computer interface (BCI) training in the future.