mlsanityData to 14 Oct 2026

Structure-Preserving Transformers for Sequences of SPD Matrices

arXiv 2309.07579 ↗First posted 14 Sep 2023cs.LG eess.SPcited by 4

Not on the map. Why?

Abstract (arXiv)

In recent years, Transformer-based auto-attention mechanisms have been successfully applied to the analysis of a variety of context-reliant data types, from texts to images and beyond, including data from non-Euclidean geometries. In this paper, we present such a mechanism, designed to classify sequences of Symmetric Positive Definite matrices while preserving their Riemannian geometry throughout the analysis. We apply our method to automatic sleep staging on timeseries of EEG-derived covariance matrices from a standard dataset, obtaining high levels of stage-wise performance.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 13 of its 29 references that are in the map. First posted on arXiv 14 Sep 2023.

Builds on

13 of 29 references.

References · 13

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  3. 2016A Riemannian Network for SPD Matrix Learningcited by 162
  4. 2017DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEGcited by 157
  5. 2020XSleepNet: Multi-View Sequential Model for Automatic Sleep Stagingcited by 88
  6. 2021SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantificationcited by 84
  7. 2022MAtt: A Manifold Attention Network for EEG Decodingcited by 44
  8. 2021Automatic Sleep Staging of EEG Signals: Recent Development, Challenges, and Future Directionscited by 43
  9. 2021Universal Approximation Under Constraints is Possible with Transformerscited by 32
  10. 2021RobustSleepNet: Transfer learning for automated sleep staging at scalecited by 24
  11. 2014From Manifold to Manifold: Geometry-Aware Dimensionality Reduction for SPD Matricescited by 22
  12. 2023L-SeqSleepNet: Whole-cycle Long Sequence Modelling for Automatic Sleep Stagingcited by 21
1 more
  1. 2022Multi-manifold Attention for Vision Transformerscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 2.

4 papers.

  1. 2025Transformer-based EEG Decoding: A Surveycited by 3
  2. 2026Subgroups of U(d) Induce Natural RNN and Transformer Architecturescited by 1
  3. 2024Automatic Classification of Sleep Stages from EEG Signals Using Riemannian Metrics and Transformer Networkscited by 0
  4. 2026A Unified SPD Token Transformer Framework for EEG Classification: Systematic Comparison of Geometric Embeddingscited by 0