mlsanityData to 14 Oct 2026

The Capability Convergence Hypothesis: Capability from Access Structure, Not Scale

arXiv 2607.14144 ↗First posted 14 Jul 2026cs.AI cs.IT math.ITcited by 4

Not on the map. Why?

Abstract (arXiv)

The Platonic Representation Hypothesis (PRH) holds that as models scale, representations of heterogeneous networks converge toward a shared model of reality. We propose its sequel and boundary, the Capability Convergence Hypothesis (CCH): under a fixed per-token inference budget, representational convergence does not entail capability convergence. Capability instead converges toward a class, the access-complete hybrid: any architecture holding both a compressive O(1)-state channel and a scalable verbatim-index channel. We anchor it on a witness task, the Newton's-apple problem in an infinite stream, and name three resource walls: a Shannon wall barring any o(Nb)-state architecture, a horizon wall barring any fixed window, and a circuit wall barring fixed-depth attention-only composition (conditional on TC0 != NC1). Under an explicit separability assumption a hybrid crosses all three by paying each wall's price, so capability is strictly super-additive under composition. We separate what we prove from what we conjecture: the access-completeness principle rests on information-theoretic lower bounds and pre-registered experiments, while the field-level convergence trend is an economics-motivated conjecture. We report the first pre-registered small-scale tests under criteria frozen before the data: the predicted scissors gap is measured (exact-retrieval error 0.994 vs. 0.000 once a 64-scalar state gains one global-attention layer), the state-tracking bifurcation lands at the registered boundary, and a conjunction witness shows an irreducibly two-channel solution; one prediction failed with its direction reversed and is reported as such. Representational convergence is given freely by scale; capability convergence must be purchased by access structure.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 47 of its 62 references that are in the map. First posted on arXiv 14 Jul 2026.

Builds on

47 of 62 references.

References · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  5. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  7. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  8. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  9. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  10. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  11. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  12. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
35 more
  1. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 613
  2. 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 514
  3. 2020Hopfield Networks is All You Needcited by 475
  4. 2024Jamba: A Hybrid Transformer-Mamba Language Modelcited by 401
  5. 2024The Platonic Representation Hypothesiscited by 382
  6. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 260
  7. 2024Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Modelscited by 242
  8. 2023Language Modeling Is Compressioncited by 225
  9. 2024Repeat After Me: Transformers are Better than State Space Models at Copyingcited by 216
  10. 2024Simple linear attention language models balance the recall-throughput tradeoffcited by 208
  11. 2023Zoology: Measuring and Improving Recall in Efficient Language Modelscited by 204
  12. 2025Kimi Linear: An Expressive, Efficient Attention Architecturecited by 196
  13. 2022The Parallelism Tradeoff: Limitations of Log-Precision Transformerscited by 193
  14. 2024The Illusion of State in State-Space Modelscited by 187
  15. 2024An Empirical Study of Mamba-based Language Modelscited by 186
  16. 2024Zamba: A Compact 7B SSM Hybrid Modelcited by 124
  17. 2024The Mamba in the Llama: Distilling and Accelerating Hybrid Modelscited by 122
  18. 2024Unlocking State-Tracking in Linear RNNs Through Negative Eigenvaluescited by 101
  19. 2025Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Modelscited by 100
  20. 2024Hymba: A Hybrid-head Architecture for Small Language Modelscited by 99
  21. 2025DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Productscited by 75
  22. 2025Harnessing the Universal Geometry of Embeddingscited by 55
  23. 2024The Expressive Capacity of State Space Models: A Formal Language Perspectivecited by 46
  24. 2024The Zamba2 Suite: Technical Reportcited by 21
  25. 2024Understanding and Mitigating Bottlenecks of State Space Models through the Lens of Recency and Over-smoothingcited by 18
  26. 2026Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scalecited by 16
  27. 2025Hybrid Architectures for Language Models: Systematic Analysis and Design Insightscited by 14
  28. 2024Expansion Span: Combining Fading Memory and Retrieval in Hybrid State Space Modelscited by 11
  29. 2025Understanding and Enhancing Mamba-Transformer Hybrids for Memory Recall and Language Modelingcited by 5
  30. 2026Priming: Hybrid State Space Models From Pre-trained Transformerscited by 5
  31. 2026Rethinking the Role of Efficient Attention in Hybrid Architecturescited by 5
  32. 2025Some Attention is All You Need for Retrievalcited by 3
  33. 2026Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoningcited by 3
  34. 2026Retrievit: In-context Retrieval Capabilities of Transformers, State Space Models, and Hybrid Architecturescited by 1
  35. 2026Reasoning Primitives in Hybrid and Non-Hybrid LLMs: Do Architectural Differences Yield Advantages in State-Tracking and Recall?cited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.