mlsanityData to 7 Oct 2026

Frozen Memory Is Not Enough: Rethinking External Memory as Extraction

arXiv 2608.17050 ↗First posted 17 Aug 2026cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Methods for improving knowledge use in large language models typically fall into two regimes. Non-parametric retrieval offers flexible access to external knowledge, but adds retrieval latency, context overhead, and only shallow integration with the backbone. Parametric adaptation is efficient at inference time, but entangles knowledge with model weights and can be hard to update, audit, or transfer. Engram-style hashed memory occupies a middle regime: it stores learned information in an external, addressable table, yet consumes that table through a small learned reader. This raises a basic question: when such a memory is moved across backbones, what matters more, the frozen memory itself or the target-side reader? We study this question through cross-model frozen-memory extraction, in which a memory trained on a source model is frozen and attached to a different target model, with only a lightweight reader trained. Ablations show that learned memory content and correct addressing both matter, but the transferred table becomes useful only through a reader aligned to the target model. In downstream question answering tasks, a dual-layer, four-branch reader nearly closes the gap between same-model and cross-model reuse, achieving an average score of 38.8 under our controlled evaluation protocol. Moreover, when the provider reader is directly compatible with the target interface, the frozen artifact can provide substantial utility without target-side training, while optional reader adaptation yields further improvement. These results suggest that Engram can serve as a reusable external knowledge artifact, provided that the target has access to a compatible reader interface; target-side adaptation can further improve alignment when direct reader reuse is insufficient.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 34 of its 41 references that are in the map. First posted on arXiv 17 Aug 2026.

Builds on

34 of 41 references.

References · 34

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  6. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  7. 2016Pointer Sentinel Mixture Modelscited by 2,992
  8. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  11. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  12. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
22 more
  1. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  2. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  3. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  4. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  5. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  6. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  7. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  8. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  9. 2014Neural Turing Machinescited by 630
  10. 2024TinyLlama: An Open-Source Small Language Modelcited by 604
  11. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  12. 2024The Platonic Representation Hypothesiscited by 362
  13. 2024Titans: Learning to Memorize at Test Timecited by 303
  14. 2013Exploiting Similarities among Languages for Machine Translationcited by 187
  15. 2024Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Datasetcited by 153
  16. 2019Large Memory Layers with Product Keyscited by 120
  17. 2026Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Modelscited by 78
  18. 2024Memory Layers at Scalecited by 39
  19. 2025MLP Memory: A Retriever-Pretrained Memory for Large Language Modelscited by 19
  20. 2025Transferring Linear Features Across Language Models With Model Stitchingcited by 17
  21. 2025LoRA-X: Bridging Foundation Models with Training-Free Cross-Model Adaptationcited by 10
  22. 2025Cross-LoRA: A Data-Free LoRA Transfer Framework across Heterogeneous LLMscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.