# Pre-trained Transformer-models using chronic invasive electrophysiology for symptom decoding without patient-individual training

arXiv: https://arxiv.org/abs/2508.10160 · HTML: https://mlsanity.com/paper/2508.10160/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 20 references that are in the map.
- First posted on arXiv 13 Aug 2025.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Multimodal Token Fusion for Vision Transformers](https://mlsanity.com/paper/2204.08721/) (2022)
- [Learning to Encode Position for Transformer with Continuous Dynamical Model](https://mlsanity.com/paper/2003.09229/) (2020)
- [BrainWave: A Brain Signal Foundation Model for Clinical Applications](https://mlsanity.com/paper/2402.10251/) (2024)

## Built on by (most cited first, 1 of 1)

- [Subspace Learning with Interval-Censored Likelihoods for Dequantizing Percept PC LFP Snapshots](https://mlsanity.com/paper/2609.22658/) (2026)

## Abstract (arXiv)

Neural decoding of pathological and physiological states can enable patient-individualized closed-loop neuromodulation therapy. Recent advances in pre-trained large-scale foundation models offer the potential for generalized state estimation without patient-individual training. Here we present a foundation model trained on chronic longitudinal deep brain stimulation recordings spanning over 24 days. Adhering to long time-scale symptom fluctuations, we highlight the extended context window of 30 minutes. We present an optimized pre-training loss function for neural electrophysiological data that corrects for the frequency bias of common masked auto-encoder loss functions due to the 1-over-f power law. We show in a downstream task the decoding of Parkinson's disease symptoms with leave-one-subject-out cross-validation without patient-individual training.