Consolidator: Learning Persistent Routed Memory Across Context Boundaries
Not on the map. Why?
Abstract (arXiv)
Copying short-term memory (STM) into a slower store can preserve state across a context boundary, but persistence alone does not ensure that the retained state influences subsequent memory access. We test this distinction in a Phasor Memory Network (PMNet) using Consolidator, a shared slot-local operator that transforms routed STM before accumulating it into long-term memory (LTM), without replaying the source tokens. After each consolidation, the KV cache and STM are cleared. The retained LTM can still be read and is also fed into the hierarchical router, thereby conditioning which explicit-memory slots subsequent inputs access. We evaluate this mechanism on a two-segment modulo-10 mapping task in which the second segment updates the mapping at the same memory address. Following a second consolidation and reset, a held-out query must recover the updated mapping from LTM. The backbone and memory interface are frozen, leaving only 12.35K Consolidator parameters trainable (0.041% of a 29.95M model). Across five paired runs from the same STM-pretraining checkpoint, direct LTM routing raises updated-mapping recall from 44.38±1.94% to 87.02±1.76% (+42.64±1.10 percentage points), while immediate STM recall remains 89.90% in both conditions; both train separate Consolidators and retain the same LTM read paths. Learned consolidation outperforms forced identity accumulation by 21.40±1.91 percentage points without routing and 68.70±1.76 with routing. Thus, on this task, consolidated LTM serves as both retrievable content and an access state that shapes subsequent slot selection.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 19 references that are in the map. First posted on arXiv 12 Aug 2026.
Builds on
16 of 19 references.
References · 16
- 2017Attention Is All You Needcited by 60,361
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
- 2019Compressive Transformers for Long-Range Sequence Modellingcited by 726
- 2014Neural Turing Machinescited by 644
- 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 355
- 2024Titans: Learning to Memorize at Test Timecited by 328
- 2022Recurrent Memory Transformercited by 203
- 2024Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attentioncited by 200
- 2016Using Fast Weights to Attend to the Recent Pastcited by 170
- 2021Replay in Deep Learning: Current Approaches and Missing Biological Elementscited by 72
4 more
- 2026Auto-Dreamer: Learning Offline Memory Consolidation for Language Agentscited by 10
- 2026Context Distillation as Latent Memory Managementcited by 6
- 2026Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Trainingcited by 3
- 2026Phasor Memory Networks: Stable Backpropagation Through Time for Scalable Explicit Memorycited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Memory as a cache: Exact context reuse and deletion by construction
- 2026Multi-Head Recurrent Memory Agents
- 2026Do Dynamic Routers Need Memory? HeRo: History-Aware Routing for Efficient LLM Inference
- 2026Stop When Memory Suffices: Evidence-Conditioned Progressive Execution for LLM Agents
- 2026Just Let Linear States Forget the Distant Past: Prefix Caching via Suffix Replay for Hybrid LLMs
- 2026Dual-Layer Agentic Memory with Fast Write Routing and Slow Consolidation
- 2026CoEvo-Mem: Co-Evolving Retrieval Policy and Memory Bank for LLM Agents
- 2026All-Mem: Agentic Lifelong Memory via Dynamic Topology Evolution