# Adaptive Forgetting for Nonstationary Optimization: Towards Robust EEG Decoding

arXiv: https://arxiv.org/abs/2609.24233 · HTML: https://mlsanity.com/paper/2609.24233/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 40 references that are in the map.
- First posted on arXiv 21 Sep 2026.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training](https://mlsanity.com/paper/2305.14342/) (2023)
- [MARS: Unleashing the Power of Variance Reduction for Training Large Models](https://mlsanity.com/paper/2411.10438/) (2024)
- [TSception: Capturing Temporal Dynamics and Spatial Asymmetry from EEG for Emotion Recognition](https://mlsanity.com/paper/2104.02935/) (2021)
- [Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN](https://mlsanity.com/paper/2412.13795/) (2024)
- [EEG-Deformer: A Dense Convolutional Transformer for Brain-computer Interfaces](https://mlsanity.com/paper/2405.00719/) (2024)
- [EmT: A Novel Transformer for Generalized Cross-subject EEG Emotion Recognition](https://mlsanity.com/paper/2406.18345/) (2024)
- [Nostalgic Adam: Weighting more of the past gradients when designing the adaptive learning rate](https://mlsanity.com/paper/1805.07557/) (2018)
- [LibEER: A Comprehensive Benchmark and Algorithm Library for EEG-based Emotion Recognition](https://mlsanity.com/paper/2410.09767/) (2024)
- [Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers](https://mlsanity.com/paper/2411.00999/) (2024)
- [Dynamic GNNs for Precise Seizure Detection and Classification from EEG Data](https://mlsanity.com/paper/2405.09568/) (2024)
- [Cross-Subject Generalization for EEG Decoding: A Survey of Deep Learning Methods](https://mlsanity.com/paper/2604.27033/) (2026)
- [BiMoE: Brain-Inspired Experts for EEG-Dominant Affective State Recognition](https://mlsanity.com/paper/2603.29205/) (2026)

## Abstract (arXiv)

Electroencephalography (EEG) provides non-invasive monitoring of brain activity and is widely used in emotion recognition, motor imagery and sleep staging. Although within-subject decoding has achieved considerable progress, cross-subject generalization remains a central challenge in practical applications. EEG decoders are typically trained with Adam/AdamW under a fixed second-moment decay coefficient, even though cross-subject learning involves low signal-to-noise ratios, subject variability, and gradient nonstationarity. A fixed coefficient implicitly assumes that gradient statistics are homogeneous across layers and time, which can limit model's adaptability to cross-subject EEG signals and degrade generalization. To address these issues, we propose AFOR, a tensor-wise adaptive optimizer that converts the fixed second-moment decay coefficient into a dynamic coefficient estimated online from local gradient state. AFOR combines a Residual-Alignment Signal Scorer (RASS) and an Adaptive Forgetting Controller (AFC). RASS summarizes local gradient residuals and directional agreement into a signal-quality score, and AFC maps this score through self-referential normalization to a bounded per-step decay coefficient, with cumulative-product initialization correction maintaining consistency under time-varying decay. Under a strict cross-subject protocol on three EEG benchmarks that cover three representative fields, AFOR achieves the best average performance among the compared optimizers, improving the mean test accuracy over Adam by 3.00%, 2.07%, and 4.38%, respectively.