Memory as Resonance: A Biomimetic Architecture for Infinite Context Memory on Ergodic Phonetic Manifolds
Not on the map. Why?
Abstract (arXiv)
The memory of contemporary Large Language Models is bound by a physical paradox: as they learn, they fill up. The linear accumulation (O(N)) of Key-Value states treats context as a warehouse of static artifacts, eventually forcing a destructive choice between amnesia and latency. We challenge this discrete orthodoxy, proposing that long-term memory is not the storage of items, but the persistence of a trajectory. We introduce Phonetic Trajectory Memory (PTM), a neuro-symbolic architecture that encodes language not as a sequence of tensors, but as a continuous path on an ergodic manifold governed by irrational rotation matrices. By decoupling the navigation (an invariant O(1) geometric signal) from the reconstruction (a probabilistic generative act), PTM achieves a compression magnitude of greater than 3,000x relative to dense caches. We demonstrate that retrieval becomes a process of resonance: the phonetic trace stabilizes the model against hallucination via "Signal Consensus" mechanism, securing up to approximately 92% factual accuracy. While this aggressive abstraction alters generative texture, it unlocks immediate access latency (approximately 34ms) independent of depth. Our results suggest that infinite context does not require infinite silicon; it requires treating memory not as data to be stored, but as a reconstructive process acting on a conserved, undying physical signal.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 38 of its 65 references that are in the map. First posted on arXiv 23 Dec 2025.
Builds on
38 of 65 references.
References · 38
- 2017Attention Is All You Needcited by 59,932
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2020Longformer: The Long-Document Transformercited by 3,362
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
- 2020Big Bird: Transformers for Longer Sequencescited by 1,673
- 2020Reformer: The Efficient Transformercited by 1,658
- 2020Rethinking Attention with Performerscited by 1,584
26 more
- 2019Generating Long Sequences with Sparse Transformerscited by 1,523
- 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
- 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
- 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
- 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
- 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 448
- 2024Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attentioncited by 194
- 2023Compressing Context to Enhance Inference Efficiency of Large Language Modelscited by 185
- 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAGcited by 122
- 2025From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMscited by 119
- 2023Scaling Transformer to 1M tokens and beyond with RMTcited by 97
- 2021Sparse is Enough in Scaling Transformerscited by 86
- 2021Going Beyond Linear Transformers with Recurrent Fast Weight Programmerscited by 84
- 2024Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Modelscited by 71
- 2023Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Studycited by 63
- 2024Scaling Retrieval-Based Language Models with a Trillion-Token Datastorecited by 54
- 2024Vision Mamba: A Comprehensive Survey and Taxonomycited by 53
- 2024On the Role of Attention Masks and LayerNorm in Transformerscited by 45
- 2021Not All Memories are Created Equal: Learning to Forget by Expiringcited by 32
- 2022SoundChoice: Grapheme-to-Phoneme Models with Semantic Disambiguationcited by 14
- 2024UncertaintyRAG: Span-Level Uncertainty Enhanced Long-Context Modeling for Retrieval-Augmented Generationcited by 14
- 2025Memory-Augmented Transformers: A Systematic Review from Neuroscience Principles to Enhanced Model Architecturescited by 9
- 2016The Multivariate Generalised von Mises distribution: Inference and applicationscited by 3
- 2024UniMem: Towards a Unified View of Long-Context Large Language Modelscited by 3
- 2023Gaussian Process on the Product of Directional Manifoldscited by 2
- 2019Correct Approximation of IEEE 754 Floating-Point Arithmetic for Program Verificationcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.