The Phasor Transformer: Resolving Attention Bottlenecks on the Unit Circle
Not on the map. Why?
Abstract (arXiv)
Transformer models have redefined sequence learning, yet dot-product self-attention introduces a quadratic token-mixing bottleneck for long-context time-series. We introduce the Phasor Transformer block, a phase-native alternative representing sequence states on the unit-circle manifold S1. Each block combines lightweight trainable phase-shifts with parameter-free Discrete Fourier Transform (DFT) token coupling, achieving global O(Nlog N) mixing without explicit attention maps. Stacking these blocks defines the Large Phasor Model (LPM). We validate LPM on autoregressive time-series prediction over synthetic multi-frequency benchmarks against honest baselines: it beats a zero-parameter persistence baseline and, with the corrected gradient path, improves monotonically with depth before saturating, while remaining competitive-but-not-superior to self-attention at a fraction of the parameter count. Our results establish an explicit efficiency--accuracy frontier, showing that scalable temporal modeling in oscillatory domains can emerge from geometry-constrained phase computation with deterministic global coupling.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 24 references that are in the map. First posted on arXiv 18 Mar 2026.
Builds on
21 of 24 references.
References · 21
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
- 2020Longformer: The Long-Document Transformercited by 3,413
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
- 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
- 2020Big Bird: Transformers for Longer Sequencescited by 1,684
- 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
9 more
- 2020Rethinking Attention with Performerscited by 1,601
- 2019Generating Long Sequences with Sparse Transformerscited by 1,546
- 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
- 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
- 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
- 2020Efficient Transformers: A Surveycited by 790
- 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 761
- 2021FNet: Mixing Tokens with Fourier Transformscited by 347
- 2026PhasorFlow: A Python Library for Unit Circle Based Computingcited by 2
Built on by
Not cited yet.