Frozen Memory Is Not Enough: Rethinking External Memory as Extraction
Not on the map. Why?
Abstract (arXiv)
Methods for improving knowledge use in large language models typically fall into two regimes. Non-parametric retrieval offers flexible access to external knowledge, but adds retrieval latency, context overhead, and only shallow integration with the backbone. Parametric adaptation is efficient at inference time, but entangles knowledge with model weights and can be hard to update, audit, or transfer. Engram-style hashed memory occupies a middle regime: it stores learned information in an external, addressable table, yet consumes that table through a small learned reader. This raises a basic question: when such a memory is moved across backbones, what matters more, the frozen memory itself or the target-side reader? We study this question through cross-model frozen-memory extraction, in which a memory trained on a source model is frozen and attached to a different target model, with only a lightweight reader trained. Ablations show that learned memory content and correct addressing both matter, but the transferred table becomes useful only through a reader aligned to the target model. In downstream question answering tasks, a dual-layer, four-branch reader nearly closes the gap between same-model and cross-model reuse, achieving an average score of 38.8 under our controlled evaluation protocol. Moreover, when the provider reader is directly compatible with the target interface, the frozen artifact can provide substantial utility without target-side training, while optional reader adaptation yields further improvement. These results suggest that Engram can serve as a reusable external knowledge artifact, provided that the target has access to a compatible reader interface; target-side adaptation can further improve alignment when direct reader reuse is insufficient.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 34 of its 41 references that are in the map. First posted on arXiv 17 Aug 2026.
Builds on
34 of 41 references.
References · 34
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
- 2016Pointer Sentinel Mixture Modelscited by 2,992
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
- 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
- 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
- 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
22 more
- 2019Similarity of Neural Network Representations Revisitedcited by 1,695
- 2021Improving language models by retrieving from trillions of tokenscited by 1,278
- 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
- 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
- 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
- 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
- 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
- 2017Crowdsourcing Multiple Choice Science Questionscited by 742
- 2014Neural Turing Machinescited by 630
- 2024TinyLlama: An Open-Source Small Language Modelcited by 604
- 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
- 2024The Platonic Representation Hypothesiscited by 362
- 2024Titans: Learning to Memorize at Test Timecited by 303
- 2013Exploiting Similarities among Languages for Machine Translationcited by 187
- 2024Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Datasetcited by 153
- 2019Large Memory Layers with Product Keyscited by 120
- 2026Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Modelscited by 78
- 2024Memory Layers at Scalecited by 39
- 2025MLP Memory: A Retriever-Pretrained Memory for Large Language Modelscited by 19
- 2025Transferring Linear Features Across Language Models With Model Stitchingcited by 17
- 2025LoRA-X: Bridging Foundation Models with Training-Free Cross-Model Adaptationcited by 10
- 2025Cross-LoRA: A Data-Free LoRA Transfer Framework across Heterogeneous LLMscited by 9
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Portable Semantics, Private Dialects: Reuse and Negative Transfer in Latent Communication Between Language-Model Cellscited by 1
- 2026LatentPort: Beyond KV Cache - Cross-Model Transfer of Recurrent Memory in Hybrid Language Models: A 4B-to-9B Hybrid-State Handoff Without Target Prefix Replaycited by 0
- 2026MemoryAthena: Adaptive Routing over Latent and Generated Memoriescited by 0
- 2026MemoryAthena: Adaptive Routing over Latent and Generated Memories22 Sep 2026
- 2026Portable Semantics, Private Dialects: Reuse and Negative Transfer in Latent Communication Between Language-Model Cells10 Sep 2026
- 2026LatentPort: Beyond KV Cache - Cross-Model Transfer of Recurrent Memory in Hybrid Language Models: A 4B-to-9B Hybrid-State Handoff Without Target Prefix Replay7 Sep 2026