Phase Structure in Rotary Attention: A Spectral Framework for Semantic Continuity and Execution-Boundary Governance
Not on the map. Why?
Abstract (arXiv)
Transformer language models are usually analyzed through vector geometry, yet ordered context and rotary position encoding introduce explicit phase structure into query-key interactions. This paper develops a bounded spectral framework for examining rotary phase alignment, hidden-state continuity, and semantic drift without treating language models as literal physical wave systems. It first identifies ordered hidden-state sequences, rather than vocabulary indices, as valid domains for spectral decomposition. It then derives the Rotary Position Embedding (RoPE) attention score as a sum of magnitude-weighted cosine terms and proves a local stability lemma: uniformly bounded phase displacement limits degradation of the corresponding pre-softmax score. To extend phase analysis beyond native RoPE coordinates, the paper defines complex modal coordinates over fixed orthonormal direction pairs and introduces a weighted coherence functional for hidden-state trajectories. These constructions support a strict distinction between representational continuity and execution-boundary admissibility. Internal coherence may describe preservation of task-relevant relations, but it cannot authorize a consequential transition. Positioned against existing geometric, spectral, phase-modulation, representation-analysis, and mechanistic-interpretability accounts, the framework contributes a theoretical and methodological program for determining when spectral structure explains continuity and when governance must remain an external predicate over execution.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 7 of its 16 references that are in the map. First posted on arXiv 28 Jul 2026.
Builds on
7 of 16 references.
References · 7
- 2017Attention Is All You Needcited by 60,361
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2019Similarity of Neural Network Representations Revisitedcited by 1,755
- 2022In-context Learning and Induction Headscited by 964
- 2024Round and Round We Go! What makes Rotary Positional Encodings useful?cited by 115
- 2026Rotary Positional Embeddings as Phase Modulation: Theoretical Bounds on the RoPE Base for Long-Context Transformerscited by 7
- 2025Deconstructing Positional Information: From Attention Logits to Training Biasescited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.