The Capability Convergence Hypothesis: Capability from Access Structure, Not Scale
Not on the map. Why?
Abstract (arXiv)
The Platonic Representation Hypothesis (PRH) holds that as models scale, representations of heterogeneous networks converge toward a shared model of reality. We propose its sequel and boundary, the Capability Convergence Hypothesis (CCH): under a fixed per-token inference budget, representational convergence does not entail capability convergence. Capability instead converges toward a class, the access-complete hybrid: any architecture holding both a compressive O(1)-state channel and a scalable verbatim-index channel. We anchor it on a witness task, the Newton's-apple problem in an infinite stream, and name three resource walls: a Shannon wall barring any o(Nb)-state architecture, a horizon wall barring any fixed window, and a circuit wall barring fixed-depth attention-only composition (conditional on TC0 != NC1). Under an explicit separability assumption a hybrid crosses all three by paying each wall's price, so capability is strictly super-additive under composition. We separate what we prove from what we conjecture: the access-completeness principle rests on information-theoretic lower bounds and pre-registered experiments, while the field-level convergence trend is an economics-motivated conjecture. We report the first pre-registered small-scale tests under criteria frozen before the data: the predicted scissors gap is measured (exact-retrieval error 0.994 vs. 0.000 once a 64-scalar state gains one global-attention layer), the state-tracking bifurcation lands at the registered boundary, and a conjunction witness shows an irreducibly two-channel solution; one prediction failed with its direction reversed and is reported as such. Representational convergence is given freely by scale; capability convergence must be purchased by access structure.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 47 of its 62 references that are in the map. First posted on arXiv 14 Jul 2026.
Builds on
47 of 62 references.
References · 47
- 2017Attention Is All You Needcited by 60,361
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
- 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
- 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
- 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
35 more
- 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 613
- 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 514
- 2020Hopfield Networks is All You Needcited by 475
- 2024Jamba: A Hybrid Transformer-Mamba Language Modelcited by 401
- 2024The Platonic Representation Hypothesiscited by 382
- 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 260
- 2024Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Modelscited by 242
- 2023Language Modeling Is Compressioncited by 225
- 2024Repeat After Me: Transformers are Better than State Space Models at Copyingcited by 216
- 2024Simple linear attention language models balance the recall-throughput tradeoffcited by 208
- 2023Zoology: Measuring and Improving Recall in Efficient Language Modelscited by 204
- 2025Kimi Linear: An Expressive, Efficient Attention Architecturecited by 196
- 2022The Parallelism Tradeoff: Limitations of Log-Precision Transformerscited by 193
- 2024The Illusion of State in State-Space Modelscited by 187
- 2024An Empirical Study of Mamba-based Language Modelscited by 186
- 2024Zamba: A Compact 7B SSM Hybrid Modelcited by 124
- 2024The Mamba in the Llama: Distilling and Accelerating Hybrid Modelscited by 122
- 2024Unlocking State-Tracking in Linear RNNs Through Negative Eigenvaluescited by 101
- 2025Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Modelscited by 100
- 2024Hymba: A Hybrid-head Architecture for Small Language Modelscited by 99
- 2025DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Productscited by 75
- 2025Harnessing the Universal Geometry of Embeddingscited by 55
- 2024The Expressive Capacity of State Space Models: A Formal Language Perspectivecited by 46
- 2024The Zamba2 Suite: Technical Reportcited by 21
- 2024Understanding and Mitigating Bottlenecks of State Space Models through the Lens of Recency and Over-smoothingcited by 18
- 2026Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scalecited by 16
- 2025Hybrid Architectures for Language Models: Systematic Analysis and Design Insightscited by 14
- 2024Expansion Span: Combining Fading Memory and Retrieval in Hybrid State Space Modelscited by 11
- 2025Understanding and Enhancing Mamba-Transformer Hybrids for Memory Recall and Language Modelingcited by 5
- 2026Priming: Hybrid State Space Models From Pre-trained Transformerscited by 5
- 2026Rethinking the Role of Efficient Attention in Hybrid Architecturescited by 5
- 2025Some Attention is All You Need for Retrievalcited by 3
- 2026Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoningcited by 3
- 2026Retrievit: In-context Retrieval Capabilities of Transformers, State Space Models, and Hybrid Architecturescited by 1
- 2026Reasoning Primitives in Hybrid and Non-Hybrid LLMs: Do Architectural Differences Yield Advantages in State-Tracking and Recall?cited by 1
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Budget-Constrained Embodied Perception: Four Resource Walls and a Pre-Registered Evaluation of Access-Structured Perception on Open Models at less than 31Bcited by 0
- 2026Self-Cleaning and Captured Anyway: One Measured Primitive for Error in a Store an Agent Writes to Itself, and What a Falling Score Actually Measurescited by 0
- 2026The Free-Recipe Limit: Every Recipe Effect Measures Which Premise of an Idealised Learner Brokecited by 0
- 2026Conflicting Supervision Moves Commitment, Not Capability: A 12.29σ arrangement effect that is exactly zero under a convention-agnostic scorecited by 0
- 2026Conflicting Supervision Moves Commitment, Not Capability: A 12.29σ arrangement effect that is exactly zero under a convention-agnostic score23 Sep 2026
- 2026Self-Cleaning and Captured Anyway: One Measured Primitive for Error in a Store an Agent Writes to Itself, and What a Falling Score Actually Measures6 Sep 2026
- 2026Budget-Constrained Embodied Perception: Four Resource Walls and a Pre-Registered Evaluation of Access-Structured Perception on Open Models at less than 31B24 Aug 2026
- 2026The Free-Recipe Limit: Every Recipe Effect Measures Which Premise of an Idealised Learner Broke9 Aug 2026