# Generalizability Under Sensor Failure: Tokenization + Transformers Enable More Robust Latent Spaces

arXiv: https://arxiv.org/abs/2402.18546 · HTML: https://mlsanity.com/paper/2402.18546/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 18 references that are in the map.
- First posted on arXiv 28 Feb 2024.

## Builds on (most cited first, 5 of 5)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysis](https://mlsanity.com/paper/2402.16412/) (2024)
- [Deep Neural Imputation: A Framework for Recovering Incomplete Brain Recordings](https://mlsanity.com/paper/2206.08094/) (2022)

## Built on by (most cited first, 2 of 2)

- [Population Transformer: Learning Population-level Representations of Neural Activity](https://mlsanity.com/paper/2406.03044/) (2024)
- [Learning Time-Scale Invariant Population-Level Neural Representations](https://mlsanity.com/paper/2511.13022/) (2025)

## Abstract (arXiv)

A major goal in neuroscience is to discover neural data representations that generalize. This goal is challenged by variability along recording sessions (e.g. environment), subjects (e.g. varying neural structures), and sensors (e.g. sensor noise), among others. Recent work has begun to address generalization across sessions and subjects, but few study robustness to sensor failure which is highly prevalent in neuroscience experiments. In order to address these generalizability dimensions we first collect our own electroencephalography dataset with numerous sessions, subjects, and sensors, then study two time series models: EEGNet (Lawhern et al., 2018) and TOTEM (Talukder et al., 2024). EEGNet is a widely used convolutional neural network, while TOTEM is a discrete time series tokenizer and transformer model. We find that TOTEM outperforms or matches EEGNet across all generalizability cases. Finally through analysis of TOTEM's latent codebook we observe that tokenization enables generalization.