# ManifoldFormer: Geometric Deep Learning for Neural Dynamics on Riemannian Manifolds

arXiv: https://arxiv.org/abs/2511.16828 · HTML: https://mlsanity.com/paper/2511.16828/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 21 references that are in the map.
- First posted on arXiv 20 Nov 2025.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data](https://mlsanity.com/paper/2101.12037/) (2021)
- [CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding](https://mlsanity.com/paper/2412.07236/) (2024)
- [Neuro-GPT: Towards A Foundation Model for EEG](https://mlsanity.com/paper/2311.03764/) (2023)
- [BrainBERT: Self-supervised representation learning for intracranial recordings](https://mlsanity.com/paper/2302.14367/) (2023)
- [EEGFormer: Towards Transferable and Interpretable Large-Scale EEG Foundation Model](https://mlsanity.com/paper/2401.10278/) (2024)
- [FoME: A Foundation Model for EEG using Adaptive Temporal-Lateral Attention Scaling](https://mlsanity.com/paper/2409.12454/) (2024)
- [Large Cognition Model: Towards Pretrained EEG Foundation Model](https://mlsanity.com/paper/2502.17464/) (2025)

## Built on by (most cited first, 4 of 4)

- [SPDM: Geometry-Modulated State Space Modeling with Manifold Constraints for Time Series Forecasting](https://mlsanity.com/paper/2606.09917/) (2026)
- [An Elastic Shape Variational Autoencoder for Skeleton Pose Trajectories](https://mlsanity.com/paper/2605.09231/) (2026)
- [Structured Latent Space Modeling over Multi-Scale Temporal Patches for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2607.19404/) (2026)
- [Riemannian Attention Mechanisms for Transformers: A Theoretical Framework and Architecture Design](https://mlsanity.com/paper/2608.01283/) (2026)

## Abstract (arXiv)

Existing EEG foundation models mainly treat neural signals as generic time series in Euclidean space, ignoring the intrinsic geometric structure of neural dynamics that constrains brain activity to low-dimensional manifolds. This fundamental mismatch between model assumptions and neural geometry limits representation quality and cross-subject generalization. ManifoldFormer addresses this limitation through a novel geometric deep learning framework that explicitly learns neural manifold representations. The architecture integrates three key innovations: a Riemannian VAE for manifold embedding that preserves geometric structure, a geometric Transformer with geodesic-aware attention mechanisms operating directly on neural manifolds, and a dynamics predictor leveraging neural ODEs for manifold-constrained temporal evolution. Extensive evaluation across four public datasets demonstrates substantial improvements over state-of-the-art methods, with 4.6-4.8% higher accuracy and 6.2-10.2% higher Cohen's Kappa, while maintaining robust cross-subject generalization. The geometric approach reveals meaningful neural patterns consistent with neurophysiological principles, establishing geometric constraints as essential for effective EEG foundation models.