# Spectral Considerations on Unsupervised Time-domain EEG Representation Learning with Deep Networks

arXiv: https://arxiv.org/abs/2610.16264 · HTML: https://mlsanity.com/paper/2610.16264/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 57 references that are in the map.
- First posted on arXiv 12 Oct 2026.

## Builds on (most cited first, 17 of 17)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Tutorial on Variational Autoencoders](https://mlsanity.com/paper/1606.05908/) (2016)
- [Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models](https://mlsanity.com/paper/2501.01423/) (2025)
- [Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks](https://mlsanity.com/paper/1901.06523/) (2019)
- [REVE: A Foundation Model for EEG -- Adapting to Any Setup with Large-Scale Pretraining on 25,000 Subjects](https://mlsanity.com/paper/2510.21585/) (2025)
- [Spectral Bias in Practice: The Role of Function Frequency in Generalization](https://mlsanity.com/paper/2110.02424/) (2021)
- [Explicitly Minimizing the Blur Error of Variational Autoencoders](https://mlsanity.com/paper/2304.05939/) (2023)
- [Mitigating Spectral Bias in Neural Operators via High-Frequency Scaling for Physical Systems](https://mlsanity.com/paper/2503.13695/) (2025)
- [Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls](https://mlsanity.com/paper/2510.01631/) (2025)
- [The Spectral Bias of the Deep Image Prior](https://mlsanity.com/paper/1912.08905/) (2019)
- [IC-U-Net: A U-Net-based Denoising Autoencoder Using Mixtures of Independent Components for Automatic EEG Artifact Removal](https://mlsanity.com/paper/2111.10026/) (2021)
- [Toward Diffusible High-Dimensional Latent Spaces: A Frequency Perspective](https://mlsanity.com/paper/2511.22249/) (2025)
- [Simpler is better: spectral regularization and up-sampling techniques for variational autoencoders](https://mlsanity.com/paper/2201.07544/) (2022)
- [Denoising Vision Transformer Autoencoder with Spectral Self-Regularization](https://mlsanity.com/paper/2511.12633/) (2025)
- [EEG Artifact Detection and Correction with Deep Autoencoders](https://mlsanity.com/paper/2502.08686/) (2025)

## Abstract (arXiv)

Electroencephalography (EEG) is a non-invasive, scalable source of quantitative biomarkers for the early diagnosis of brain disorders, and deep representation learning has become a powerful paradigm for extracting information-rich latent features from raw signals. However, neural networks tend to under-represent high-frequency information, a concern that is especially acute for EEG given its oscillatory nature and characteristic 1/f^chi power decay, which risks masking disease-relevant high-frequency signatures. In this study, we systematically investigate the spectral constraints of unsupervised time-domain feature extraction across three autoencoder architectures (LSTMVAE, VAEEG, Transformer) and a state-of-the-art EEG foundation model. Using synthetic experiments and empirical validation, we show that while autoencoders capture macro-scale shifts such as alpha slowing, they remain biased against higher frequencies due to a magnitude-driven masking effect induced by the Mean Squared Error loss under 1/f-decaying spectra. Increasing latent dimensionality mitigates this attenuation but sacrifices latent compactness and generative stability. We argue that, as the field advances toward large-scale foundation-level feature extractors, it is imperative to move beyond simple time-domain reconstruction and to integrate spectral-domain loss terms and gradient-balancing techniques, so that the subtle high-frequency transients critical for diagnosing complex brain disorders are preserved in the latent representation.