# Time Window Optimization for Motor Imagery Classification via Temporal Structure Embedded Reinforcement Learning

arXiv: https://arxiv.org/abs/2610.16921 · HTML: https://mlsanity.com/paper/2610.16921/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 35 references that are in the map.
- First posted on arXiv 12 Oct 2026.

## Placed in

- Deep reinforcement learning methods

## Builds on (most cited first, 10 of 10)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [A Closer Look at Invalid Action Masking in Policy Gradient Algorithms](https://mlsanity.com/paper/2006.14171/) (2020)
- [MOABB: Trustworthy algorithm benchmarking for BCIs](https://mlsanity.com/paper/1805.06427/) (2018)
- [TSLANet: Rethinking Transformers for Time Series Representation Learning](https://mlsanity.com/paper/2404.08472/) (2024)
- [FBCNet: A Multi-view Convolutional Neural Network for Brain-Computer Interface](https://mlsanity.com/paper/2104.01233/) (2021)
- [MAtt: A Manifold Attention Network for EEG Decoding](https://mlsanity.com/paper/2210.01986/) (2022)
- [Excluding the Irrelevant: Focusing Reinforcement Learning through Continuous Action Masking](https://mlsanity.com/paper/2406.03704/) (2024)
- [CSP-Net: Common Spatial Pattern Empowered Neural Networks for EEG-Based Motor Imagery Classification](https://mlsanity.com/paper/2411.11879/) (2024)
- [Motor Imagery Classification for Asynchronous EEG-Based Brain-Computer Interfaces](https://mlsanity.com/paper/2412.09006/) (2024)

## Abstract (arXiv)

Online motor imagery-based brain-computer interfaces (MI-BCIs) typically decode continuous EEG using fixed-length sliding windows that may be misaligned with neural events. Existing window optimization methods often evaluate windows independently, overlooking temporal dependencies and potentially producing fragmented or physiologically implausible selections. We propose Temporal Structure Embedded Reinforcement Learning (TSE-RL), a framework that incorporates prior knowledge of MI event structure into a constrained Markov decision process for adaptive window optimization. First, TSE-RL assigns each window an explicit semantic role through Discard, Onset, Sustain, or Offset actions, extending binary retain/discard decisions to event-level alignment. Second, a finite-state machine masks invalid actions to enforce the temporal order of MI phases and promote physiologically plausible window sequences. Third, a Transformer-based history encoder captures dependencies across windows, improving training stability and computational efficiency compared with recurrent architectures. TSE-RL is plug-and-play and can be integrated with diverse MI classifiers. Experiments on multiple benchmark datasets demonstrate consistent improvements in decoding performance in both cross-session and cross-subject settings. Visualization analyses further suggest that the learned window selection policy aligns with neurophysiological principles.