# The RG-Flow Transformer: Encoding Scale-Free Dynamics in Scarce EEG

arXiv: https://arxiv.org/abs/2607.11950 · HTML: https://mlsanity.com/paper/2607.11950/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 18 references that are in the map.
- First posted on arXiv 11 Jul 2026.

## Builds on (most cited first, 5 of 5)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Why does deep and cheap learning work so well?](https://mlsanity.com/paper/1608.08225/) (2016)
- [An exact mapping between the Variational Renormalization Group and Deep Learning](https://mlsanity.com/paper/1410.3831/) (2014)
- [Neural Network Renormalization Group](https://mlsanity.com/paper/1802.02840/) (2018)
- [Mutual Information, Neural Networks and the Renormalization Group](https://mlsanity.com/paper/1704.06279/) (2017)

## Abstract (arXiv)

Brain field potentials are scale-free: their power spectra follow a 1/fβ law whose aperiodic exponent β tracks cortical state, and sleep depth in particular is a shift in β. We ask whether a transformer endowed with an explicit renormalization-group (RG) inductive bias the RG-Flow Transformer, which couples ordinary self-attention to a scale-aware stream with a learnable anomalous dimension γ, block-spin coarse-graining, and an entropy-gated synchronization bridge has an advantage over a parameter-matched vanilla transformer on real, scarce EEG. Using the PhysioNet Sleep-EDF corpus with a strict leakage-free by-subject hold-out, we (i) benchmark RG-Flow against a param-matched vanilla transformer and a hierarchy-only ablation on 5-class AASM sleep staging, (ii) sweep the per-subject data budget to look for the inductive-bias crossover predicted when data are scarce, and (iii) test whether RG-Flow's learned γ tracks the measured spectral exponent β out-of-sample a quantity the vanilla model does not possess. Across 5 subjects and 5 seeds under leave-one-subject-out cross-validation, RG-Flow and the vanilla transformer are statistically indistinguishable on 5-class staging (77.3% vs 77.0% accuracy; paired p=0.294), and the predicted scarce-data crossover does not appear: vanilla is numerically ahead at every data-limited budget. What does separate the models is interpretability RG-Flow recovers the continuous spectral exponent out-of-sample (β-recovery R2 = 0.416), a capability the vanilla architecture has no analogue for.