mlsanityData to 7 Oct 2026

The RG-Flow Transformer: Encoding Scale-Free Dynamics in Scarce EEG

arXiv 2607.11950 ↗First posted 11 Jul 2026cs.LG cs.AI q-bio.NCcited by 0

Not on the map. Why?

Abstract (arXiv)

Brain field potentials are scale-free: their power spectra follow a 1/fβ law whose aperiodic exponent β tracks cortical state, and sleep depth in particular is a shift in β. We ask whether a transformer endowed with an explicit renormalization-group (RG) inductive bias the RG-Flow Transformer, which couples ordinary self-attention to a scale-aware stream with a learnable anomalous dimension γ, block-spin coarse-graining, and an entropy-gated synchronization bridge has an advantage over a parameter-matched vanilla transformer on real, scarce EEG. Using the PhysioNet Sleep-EDF corpus with a strict leakage-free by-subject hold-out, we (i) benchmark RG-Flow against a param-matched vanilla transformer and a hierarchy-only ablation on 5-class AASM sleep staging, (ii) sweep the per-subject data budget to look for the inductive-bias crossover predicted when data are scarce, and (iii) test whether RG-Flow's learned γ tracks the measured spectral exponent β out-of-sample a quantity the vanilla model does not possess. Across 5 subjects and 5 seeds under leave-one-subject-out cross-validation, RG-Flow and the vanilla transformer are statistically indistinguishable on 5-class staging (77.3% vs 77.0% accuracy; paired p=0.294), and the predicted scarce-data crossover does not appear: vanilla is numerically ahead at every data-limited budget. What does separate the models is interpretability RG-Flow recovers the continuous spectral exponent out-of-sample (β-recovery R2 = 0.416), a capability the vanilla architecture has no analogue for.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 5 of its 18 references that are in the map. First posted on arXiv 11 Jul 2026.

Builds on

5 of 18 references.

References · 5

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016Why does deep and cheap learning work so well?cited by 100
  3. 2014An exact mapping between the Variational Renormalization Group and Deep Learningcited by 52
  4. 2018Neural Network Renormalization Groupcited by 48
  5. 2017Mutual Information, Neural Networks and the Renormalization Groupcited by 32

Built on by

Not cited yet.