# Average Rankings Mask Per-Subject Optimality: A Friedman-Nemenyi Benchmark of EEG Motor-Imagery BCI Decoders

arXiv: https://arxiv.org/abs/2606.24394 · HTML: https://mlsanity.com/paper/2606.24394/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 45 references that are in the map.
- First posted on arXiv 23 Jun 2026.

## Builds on (most cited first, 8 of 8)

- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [MOABB: Trustworthy algorithm benchmarking for BCIs](https://mlsanity.com/paper/1805.06427/) (2018)
- [Transfer Learning for Brain-Computer Interfaces: A Euclidean Space Data Alignment Approach](https://mlsanity.com/paper/1808.05464/) (2018)
- [Transfer Learning in Brain-Computer Interfaces](https://mlsanity.com/paper/1512.00296/) (2015)
- [The largest EEG-based BCI reproducibility study for open science: the MOABB benchmark](https://mlsanity.com/paper/2404.15319/) (2024)
- [Toward cross-subject and cross-session generalization in EEG-based emotion recognition: Systematic review, taxonomy, and methods](https://mlsanity.com/paper/2212.08744/) (2022)
- [Progress in Brain Computer Interfaces: Challenges and Trends](https://mlsanity.com/paper/1901.03442/) (2019)
- [EEG-DG: A Multi-Source Domain Generalization Framework for Motor Imagery EEG Classification](https://mlsanity.com/paper/2311.05415/) (2023)

## Abstract (arXiv)

Electroencephalography (EEG) is the dominant non-invasive modality for brain-computer interfaces (BCIs), yet reliable decoding of motor imagery is hampered by inter- and intra-individual variability. A recurring claim is that one decoding pipeline, most often a spatial or Riemannian method, is broadly preferable. We test the weakest version of that claim under the most favourable conditions. Using the Mother of All BCI Benchmarks (MOABB) framework, we evaluated 1,056 decoding configurations (feature extractor x scaler x classifier), >340,000 subject-level model fits, across three public left-versus-right motor-imagery datasets (PhysionetMI, 109 participants; Cho2017, 52; Zhou2016, 4) and two frequency bands (8-15 Hz, 8-30 Hz). Every model is fit and tested within a single session of a single participant, the easiest regime, giving every pipeline its best chance. We apply the statistics standard for multi-classifier comparison: Friedman omnibus tests, Nemenyi critical-difference analysis and Wilcoxon signed-rank tests with effect sizes. Covariance tangent-space projection (cov-tgsp) and Common Spatial Patterns (CSP) are the strongest families, but their ordering is dataset-dependent and, on the largest and most heterogeneous cohort (PhysionetMI), statistically indistinguishable (Nemenyi p = 0.27; Kendall's W = 0.11). At the individual level the single best pipeline is optimal for only 35% of PhysionetMI participants, and nonlinear descriptors are best for roughly one third; matching pipeline to participant adds about seven accuracy points over the best fixed choice. The ranking is not an artefact of dimensionality, and classifier and scaler choices are secondary to the feature representation. Even in the easiest regime, no single pipeline dominates: a lower bound on the personalization problem and a quantitative case for participant-aware model selection rather than a universal decoder.