# Anti-Periodic Positional Encoding: M"obius Boundary Conditions Make In-Context Retrieval Reliable

arXiv: https://arxiv.org/abs/2607.21405 · HTML: https://mlsanity.com/paper/2607.21405/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 18 references that are in the map.
- First posted on arXiv 23 Jul 2026.

## Builds on (most cited first, 13 of 13)

- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale](https://mlsanity.com/paper/2406.17557/) (2024)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [Extending Context Window of Large Language Models via Positional Interpolation](https://mlsanity.com/paper/2306.15595/) (2023)
- [YaRN: Efficient Context Window Extension of Large Language Models](https://mlsanity.com/paper/2309.00071/) (2023)
- [The Impact of Positional Encoding on Length Generalization in Transformers](https://mlsanity.com/paper/2305.19466/) (2023)
- [LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens](https://mlsanity.com/paper/2402.13753/) (2024)
- [Landmark Attention: Random-Access Infinite Context Length for Transformers](https://mlsanity.com/paper/2305.16300/) (2023)
- [Fourier Position Embedding: Enhancing Attention's Periodic Extension for Length Generalization](https://mlsanity.com/paper/2412.17739/) (2024)
- [Expanding Expressivity in Transformer Models with M"obiusAttention](https://mlsanity.com/paper/2409.12175/) (2024)
- [MrRoPE: Mixed-radix Rotary Position Embedding](https://mlsanity.com/paper/2601.22181/) (2026)
- [Periodic RoPE for Infinite Context LLMs](https://mlsanity.com/paper/2605.27980/) (2026)

## Abstract (arXiv)

M"obius RoPE is a rotary positional encoding built on the anti-periodic frequency ladder θi=π(2i+1)/N: every rotation plane advances by an odd multiple of π across the training context, so the positional holonomy is -1 and the two ends of the sequence are deterministically coupled through a closed-form Dirichlet "dipole"; to our knowledge this is the first anti-periodic boundary condition in positional encoding. We verify the theory numerically to ~ 10-6 and pretrain 48 models spanning six 160M-class and three 410M-class arms (2B FineWeb-Edu tokens each; the hybrid arm puts M"obius frequencies on 25% of heads). Hybrid perplexity is unchanged (29.66 vs. 29.72), but needle-in-a-haystack retrieval becomes reliable: 90.3±5.7% versus 63.3±31.4% at context 512 (n=6 seeds), observed worst seed 86% versus 14%, robust variance tests p=0.013-0.029 (unadjusted), recurring at 410M (Levene p=0.040). Matched controls isolate the mechanism: an aperiodic ladder in the same frequency band reproduces none of the effect, and a periodic (holonomy +1) ladder only a fraction. Swapping trained models' frequency table back to standard RoPE (weights frozen) collapses retrieval, with damage concentrated on far needles: trained models depend on this long-range geometry. A NoPE arm is even more reliable at short context but pays a 13% perplexity tax and extrapolates worst; only the anti-periodic hybrid pairs baseline perplexity with a high reliability floor. The effect is scoped to single-needle retrieval within the training window; a one-line frequency swap thus provides zero-cost insurance against the retrieval seed lottery.