# NeuroPB: Scaling Neural Decoding with Pretrained Behavioral Representations

arXiv: https://arxiv.org/abs/2608.04389 · HTML: https://mlsanity.com/paper/2608.04389/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 5 Aug 2026.

## Builds on (most cited first, 15 of 15)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [BridgeData V2: A Dataset for Robot Learning at Scale](https://mlsanity.com/paper/2308.12952/) (2023)
- [Perceiver IO: A General Architecture for Structured Inputs & Outputs](https://mlsanity.com/paper/2107.14795/) (2021)
- [Any-point Trajectory Modeling for Policy Learning](https://mlsanity.com/paper/2401.00025/) (2023)
- [Decoding speech perception from non-invasive brain recordings](https://mlsanity.com/paper/2208.12266/) (2022)
- [Learnable latent embeddings for joint behavioral and neural analysis](https://mlsanity.com/paper/2204.00673/) (2022)
- [Decoding Natural Images from EEG for Object Recognition](https://mlsanity.com/paper/2308.13234/) (2023)
- [Representation learning for neural population activity with Neural Data Transformers](https://mlsanity.com/paper/2108.01210/) (2021)
- [Machine learning for neural decoding](https://mlsanity.com/paper/1708.00909/) (2017)
- [Neural Encoding and Decoding at Scale](https://mlsanity.com/paper/2504.08201/) (2025)
- [NeuroBridge: Bio-Inspired Self-Supervised EEG-to-Image Decoding via Cognitive Priors and Bidirectional Semantic Alignment](https://mlsanity.com/paper/2511.06836/) (2025)
- [A cross-species neural foundation model for end-to-end speech decoding](https://mlsanity.com/paper/2511.21740/) (2025)
- [Leveraging unlabelled data for generalizable neural population decoding](https://mlsanity.com/paper/2607.14086/) (2026)

## Abstract (arXiv)

Decoding continuous motor trajectories from neural activity is essential for developing practical brain-computer interfaces (BCIs). However, current neural decoders are constrained by the limited scale and heterogeneity of neural recordings. In contrast, behavioral data can be collected more readily and at substantially larger scale from humans, animals, simulations, and robotic systems. Here, we introduce NeuroPB, a framework that scales neural decoding by transferring knowledge from pretrained behavioral representations. NeuroPB first pretrains a motor encoder on large-scale motor behavior data and then aligns neural activity with the resulting behavioral representation space using a limited set of paired neural-behavioral recordings. A neural encoder and lightweight motor decoder are subsequently optimized to reconstruct continuous movement from the aligned neural representations. Across multiple macaque motor datasets, behavioral pretraining improves trajectory decoding, including an 11% R2 increase on center-out and 8% on random-target compared with training the motor encoder from scratch. Notably, pretraining on robotic trajectories achieves performance comparable to pretraining on macaque trajectories, demonstrating that transferable kinematic structure is shared across biological and artificial models. Moreover, decoding performance improves as the scale and diversity of robotic pretraining data increase, when the amount of neural data is fixed. Pretraining also enhances generalization across recording sessions, subjects, and motor tasks, with only 10% calibration needed to match training from scratch. Overall, these results establish behavioral pretraining as a scalable source for neural decoding and provide a promising route toward high-performance and calibration-efficient BCIs under limited neural data.