# MOABB: Trustworthy algorithm benchmarking for BCIs

arXiv: https://arxiv.org/abs/1805.06427 · HTML: https://mlsanity.com/paper/1805.06427/

## Key facts

- Cited by 82 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 May 2018.

## Built on by (most cited first, 30 of 82)

- [Decoding speech perception from non-invasive brain recordings](https://mlsanity.com/paper/2208.12266/) (2022)
- [Source-Free Unsupervised Domain Adaptation: A Survey](https://mlsanity.com/paper/2301.00265/) (2022)
- [Brain decoding: toward real-time reconstruction of visual perception](https://mlsanity.com/paper/2310.19812/) (2023)
- [DreamDiffusion: Generating High-Quality Images from Brain EEG Signals](https://mlsanity.com/paper/2306.16934/) (2023)
- [SPD domain-specific batch normalization to crack interpretable unsupervised domain adaptation in EEG](https://mlsanity.com/paper/2206.01323/) (2022)
- [Data augmentation for learning predictive models on EEG: a systematic comparison](https://mlsanity.com/paper/2206.14483/) (2022)
- [WOODS: Benchmarks for Out-of-Distribution Generalization in Time Series](https://mlsanity.com/paper/2203.09978/) (2022)
- [BrainVis: Exploring the Bridge between Brain and Visual Signals via Image Reconstruction](https://mlsanity.com/paper/2312.14871/) (2023)
- [The largest EEG-based BCI reproducibility study for open science: the MOABB benchmark](https://mlsanity.com/paper/2404.15319/) (2024)
- [NeuralBench: A Unifying Framework to Benchmark NeuroAI Models](https://mlsanity.com/paper/2605.08495/) (2026)
- [Brain-Conditional Multimodal Synthesis: A Survey and Taxonomy](https://mlsanity.com/paper/2401.00430/) (2023)
- [T-TIME: Test-Time Information Maximization Ensemble for Plug-and-Play BCIs](https://mlsanity.com/paper/2412.07228/) (2024)
- [SPDIM: Source-Free Unsupervised Conditional and Label Shift Adaptation in EEG](https://mlsanity.com/paper/2411.07249/) (2024)
- [Deep Riemannian Networks for End-to-End EEG Decoding](https://mlsanity.com/paper/2212.10426/) (2022)
- [Riemannian Multinomial Logistics Regression for SPD Neural Networks](https://mlsanity.com/paper/2305.11288/) (2023)
- [EEG motor imagery decoding: A framework for comparative analysis with channel attention mechanisms](https://mlsanity.com/paper/2310.11198/) (2023)
- [Interpretable and Robust AI in EEG Systems: A Survey](https://mlsanity.com/paper/2304.10755/) (2023)
- [Evaluating the structure of cognitive tasks with transfer learning](https://mlsanity.com/paper/2308.02408/) (2023)
- [A Lie Group Approach to Riemannian Batch Normalization](https://mlsanity.com/paper/2403.11261/) (2024)
- [RMLR: Extending Multinomial Logistic Regression into General Geometries](https://mlsanity.com/paper/2409.19433/) (2024)
- [Federated Motor Imagery Classification for Privacy-Preserving Brain-Computer Interfaces](https://mlsanity.com/paper/2412.01079/) (2024)
- [Spatial Distillation based Distribution Alignment (SDDA) for Cross-Headset EEG Classification](https://mlsanity.com/paper/2503.05349/) (2025)
- [NeuroAtlas: Benchmarking Foundation Models for Clinical EEG and Brain-Computer Interfaces](https://mlsanity.com/paper/2605.14698/) (2026)
- [Functional connectivity ensemble method to enhance BCI performance (FUCONE)](https://mlsanity.com/paper/2111.03122/) (2021)
- [Minimizing subject-dependent calibration for BCI with Riemannian transfer learning](https://mlsanity.com/paper/2111.12071/) (2021)
- [Dual-TSST: A Dual-Branch Temporal-Spectral-Spatial Transformer Model for EEG Decoding](https://mlsanity.com/paper/2409.03251/) (2024)
- [Motor Imagery Decoding Using Ensemble Curriculum Learning and Collaborative Training](https://mlsanity.com/paper/2211.11460/) (2022)
- [Personalised Federated Learning On Heterogeneous Feature Spaces](https://mlsanity.com/paper/2301.11447/) (2023)
- [Classification of BCI-EEG based on augmented covariance matrix](https://mlsanity.com/paper/2302.04508/) (2023)
- [Pseudo-online framework for BCI evaluation: A MOABB perspective](https://mlsanity.com/paper/2308.11656/) (2023)

## Abstract (arXiv)

BCI algorithm development has long been hampered by two major issues: small sample sets and a lack of reproducibility. We offer a solution to both of these problems via a software suite that streamlines both the issues of finding and preprocessing data in a reliable manner, as well as that of using a consistent interface for machine learning methods. By building on recent advances in software for signal analysis implemented in the MNE toolkit, and the unified framework for machine learning offered by the scikit-learn project, we offer a system that can improve BCI algorithm development. This system is fully open-source under the BSD licence and available at https://github.com/NeuroTechX/moabb. To validate our efforts, we analyze a set of state-of-the-art decoding algorithms across 12 open access datasets, with over 250 subjects. Our analysis confirms that different datasets can result in very different results for identical processing pipelines, highlighting the need for trustworthy algorithm benchmarking in the field of BCIs, and further that many previously validated methods do not hold up when applied across different datasets, which has wide-reaching implications for practical BCIs.