# Evaluating Latent Space Robustness and Uncertainty of EEG-ML Models under Realistic Distribution Shifts

arXiv: https://arxiv.org/abs/2209.11233 · HTML: https://mlsanity.com/paper/2209.11233/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 56 references that are in the map.
- First posted on arXiv 22 Sep 2022.

## Builds on (most cited first, 21 of 21)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1901.09960/) (2019)
- [A Fine-Grained Analysis on Distribution Shift](https://mlsanity.com/paper/2110.11328/) (2021)
- [Efficient Domain Generalization via Common-Specific Low-Rank Decomposition](https://mlsanity.com/paper/2003.12815/) (2020)
- [Uncovering the structure of clinical EEG signals with self-supervised learning](https://mlsanity.com/paper/2007.16104/) (2020)
- [The reliability of a deep learning model in clinical out-of-distribution MRI data: a multicohort study](https://mlsanity.com/paper/1911.00515/) (2019)
- [An Empirical Framework for Domain Generalization in Clinical Settings](https://mlsanity.com/paper/2103.11163/) (2021)
- ["Brilliant AI Doctor" in Rural China: Tensions and Challenges in AI-Powered CDSS Deployment](https://mlsanity.com/paper/2101.01524/) (2021)
- [Tutorial: Safe and Reliable Machine Learning](https://mlsanity.com/paper/1904.07204/) (2019)
- [EEG-GCNN: Augmenting Electroencephalogram-based Neurological Disease Diagnosis using a Domain-guided Graph Convolutional Neural Network](https://mlsanity.com/paper/2011.12107/) (2020)
- [GeomCA: Geometric Evaluation of Data Representations](https://mlsanity.com/paper/2105.12486/) (2021)
- [Delaunay Component Analysis for Evaluation of Data Representations](https://mlsanity.com/paper/2202.06866/) (2022)
- [Discovering Distribution Shifts using Latent Space Representations](https://mlsanity.com/paper/2202.02339/) (2022)
- [SCORE-IT: A Machine Learning-based Tool for Automatic Standardization of EEG Reports](https://mlsanity.com/paper/2109.05694/) (2021)

## Built on by (most cited first, 16 of 16)

- [EEG Foundation Models: A Critical Review of Current Progress and Future Directions](https://mlsanity.com/paper/2507.11783/) (2025)
- [Uncertainty Quantification for Machine Learning in Healthcare: A Survey](https://mlsanity.com/paper/2505.02874/) (2025)
- [Evaluating ML Robustness in GNSS Interference Classification, Characterization & Localization](https://mlsanity.com/paper/2409.15114/) (2024)
- [Interpretable and Robust AI in EEG Systems: A Survey](https://mlsanity.com/paper/2304.10755/) (2023)
- [A Simple Review of EEG Foundation Models: Datasets, Advancements and Future Perspectives](https://mlsanity.com/paper/2504.20069/) (2025)
- [Evaluation of (Un-)Supervised Machine Learning Methods for GNSS Interference Classification with Real-World Data Discrepancies](https://mlsanity.com/paper/2503.23775/) (2025)
- [Spatial-Functional awareness Transformer-based graph archetype contrastive learning for Decoding Visual Neural Representations from EEG](https://mlsanity.com/paper/2509.24761/) (2025)
- [Scheduled Knowledge Acquisition on Lightweight Vector Symbolic Architectures for Brain-Computer Interfaces](https://mlsanity.com/paper/2403.13844/) (2024)
- [The Challenge of Out-Of-Distribution Detection in Motor Imagery BCIs](https://mlsanity.com/paper/2603.13324/) (2026)
- [Brain-inspired Computing Based on Deep Learning for Human-computer Interaction: A Review](https://mlsanity.com/paper/2312.07213/) (2023)
- [Real-Time Decorrelation-Based Anomaly Detection for Multivariate Time Series](https://mlsanity.com/paper/2507.07559/) (2025)
- [Spatial Imputation Drives Cross-Domain Alignment for EEG Classification](https://mlsanity.com/paper/2508.03437/) (2025)
- [ADAPTive Input Training for Many-to-One Pre-Training on Time-Series Classification](https://mlsanity.com/paper/2604.08398/) (2026)
- [Atoms of Thought: Universal EEG Representation Learning with Microstates](https://mlsanity.com/paper/2605.20182/) (2026)
- [RobustSeiz: An Open-Source Framework for Benchmarking the Robustness of EEG Seizure Detection Models](https://mlsanity.com/paper/2609.04007/) (2026)
- [Interpretable but Fragile? Robustness of Concept Bottlenecks under Geometric-Semantic Perturbations](https://mlsanity.com/paper/2609.38625/) (2026)

## Abstract (arXiv)

The recent availability of large datasets in bio-medicine has inspired the development of representation learning methods for multiple healthcare applications. Despite advances in predictive performance, the clinical utility of such methods is limited when exposed to real-world data. This study develops model diagnostic measures to detect potential pitfalls before deployment without assuming access to external data. Specifically, we focus on modeling realistic data shifts in electrophysiological signals (EEGs) via data transforms and extend the conventional task-based evaluations with analyses of a) the model's latent space and b) predictive uncertainty under these transforms. We conduct experiments on multiple EEG feature encoders and two clinically relevant downstream tasks using publicly available large-scale clinical EEGs. Within this experimental setting, our results suggest that measures of latent space integrity and model uncertainty under the proposed data shifts may help anticipate performance degradation during deployment.