mlsanityData to 14 Oct 2026

Emergence of Frontier Superposition: M"obius attractor and Cascade Supervision

arXiv 2605.18820 ↗First posted 12 May 2026cs.LG cs.AIcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Superposition allows Transformers to reason in depth, carrying an entire reasoning frontier in parallel through a bounded-depth forward pass instead of unrolling serial chain-of-thought tokens. While Zhu et al. (2025) hand-crafted an equal-weight breadth-first frontier in a single residual stream for graph reachability, it remained open whether gradient descent could ever find this target amidst permutation-symmetric saddles. We close this gap on Reachability-by-Superposition over ErdHos-R'enyi graphs by isolating architectural and supervisional contributions. Architecturally, we identify a M"obius attractor: under Sn-symmetry in the tree regime, layerwise dynamics reduce to a 1D M"obius map whose zero set is a codimension-one manifold of global optima containing the equal-weight superposition state. On the supervision side, we identify Cascade Supervision: a loss class whose backward pass simultaneously delivers (A) selectivity bootstrap, (B) gradient persistence across depth, and (C) per-step discrimination (e.g., Lsup and Lnode). End-to-end supervision fails condition (B) and is provably insufficient: internal gradients at layer c decay as (np)-(D-c-2)/2 in the graph fan-out and stall before the manifold is reached. Our thesis: M"obius attractor + Cascade Supervision = emergence of superposition reasoning. The parameter-free decay law predicts a final-step cosine of 0.35 vs. 0.71 (end-to-end vs. cascade) at depth D=3; experiments confirm 0.37 vs. 0.69, matching within 0.02 at every step.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 24 of its 31 references that are in the map. First posted on arXiv 12 May 2026.

Builds on

24 of 31 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Think before you speak: Training Language Models With Pause Tokens3 Oct

Other references · 23

  1. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  3. 2022Toy Models of Superpositioncited by 971
  4. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  5. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 337
  6. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 312
  7. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 260
  8. 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Stepcited by 235
  9. 2022The Parallelism Tradeoff: Limitations of Log-Precision Transformerscited by 193
  10. 2024Let's Think Dot by Dot: Hidden Computation in Transformer Language Modelscited by 191
  11. 2022Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limitcited by 187
  12. 2023Birth of a Transformer: A Memory Viewpointcited by 161
11 more
  1. 2024The pitfalls of next-token predictioncited by 160
  2. 2023SGD learning on neural networks: leap complexity and saddle-to-saddle dynamicscited by 143
  3. 2022The merged-staircase property: a necessary and nearly sufficient condition for SGD learning of sparse functions on two-layer neural networkscited by 142
  4. 2023The mechanistic basis of data dependence and abrupt learning in an in-context classification taskcited by 118
  5. 2024How Transformers Learn Causal Structure with Gradient Descentcited by 118
  6. 2023Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformercited by 102
  7. 2024Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrappingcited by 90
  8. 2024Transformers, parallel computation, and logarithmic depthcited by 79
  9. 2023Transformers as Support Vector Machinescited by 74
  10. 2023Max-Margin Token Selection in Attention Mechanismcited by 58
  11. 2023Transformers learn through gradual rank increasecited by 52

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning
  2. 2025Emergence of Superposition: Unveiling the Training Dynamics of Chain of Continuous Thought
  3. 2026Looped Reasoning Finishes Earlier Than You Think
  4. 2025FormulaOne: Measuring the Depth of Algorithmic Reasoning Beyond Competitive Programming
  5. 2026Boosting Inference with Guided Reasoning: Stochastic Exploration for Recursive Models
  6. 2025Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought
  7. 2026A Measure-Theoretic Analysis of Reasoning: Structural Generalization and Approximation Limits
  8. 2026Outcome-Based RL Provably Leads Transformers to Reason, but Only With the Right Data