mlsanityData to 14 Oct 2026

Phase Structure in Rotary Attention: A Spectral Framework for Semantic Continuity and Execution-Boundary Governance

arXiv 2607.25507 ↗First posted 28 Jul 2026cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Transformer language models are usually analyzed through vector geometry, yet ordered context and rotary position encoding introduce explicit phase structure into query-key interactions. This paper develops a bounded spectral framework for examining rotary phase alignment, hidden-state continuity, and semantic drift without treating language models as literal physical wave systems. It first identifies ordered hidden-state sequences, rather than vocabulary indices, as valid domains for spectral decomposition. It then derives the Rotary Position Embedding (RoPE) attention score as a sum of magnitude-weighted cosine terms and proves a local stability lemma: uniformly bounded phase displacement limits degradation of the corresponding pre-softmax score. To extend phase analysis beyond native RoPE coordinates, the paper defines complex modal coordinates over fixed orthonormal direction pairs and introduces a weighted coherence functional for hidden-state trajectories. These constructions support a strict distinction between representational continuity and execution-boundary admissibility. Internal coherence may describe preservation of task-relevant relations, but it cannot authorize a consequential transition. Positioned against existing geometric, spectral, phase-modulation, representation-analysis, and mechanistic-interpretability accounts, the framework contributes a theoretical and methodological program for determining when spectral structure explains continuity and when governance must remain an external predicate over execution.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 7 of its 16 references that are in the map. First posted on arXiv 28 Jul 2026.

Builds on

7 of 16 references.

References · 7

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  3. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  4. 2022In-context Learning and Induction Headscited by 964
  5. 2024Round and Round We Go! What makes Rotary Positional Encodings useful?cited by 115
  6. 2026Rotary Positional Embeddings as Phase Modulation: Theoretical Bounds on the RoPE Base for Long-Context Transformerscited by 7
  7. 2025Deconstructing Positional Information: From Attention Logits to Training Biasescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.