mlsanityData to 7 Oct 2026

TokenMem: Faithful Knowledge Injection for Frozen LLMs

arXiv 2607.22625 ↗First posted 17 Jun 2026cs.AI cs.LGcited by 0

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval-augmented generation (RAG) enhances large language models (LLMs) with external knowledge, but suffers from knowledge conflicts: when retrieved information contradicts parametric memory, the shared self-attention pathway produces unpredictable outputs. We present TokenMem, a lightweight memory system that injects knowledge into frozen LLMs through a dedicated cross-attention channel, bypassing competition with parametric memory in the residual stream. TokenMem trains only a thin gating adapter (~3-7M parameters) via a two-phase curriculum: first learning general knowledge utilization, then strengthening faithful compliance under counterfactual knowledge. In controlled experiments on five models spanning three families (Qwen3-4B/8B/14B, LLaMA-3.1-8B, OLMo-3-7B), TokenMem achieves 69-70% Knowledge Compliance (KC) on counterfactual benchmarks, compared to 20-52% for vanilla RAG, a gap of up to 49 percentage points. Ablation studies show that the two-phase curriculum is critical: removing Phase 2 collapses KC to near-zero. Mechanistic analysis reveals that the gate adapter learns a conflict-aware, layer-specific injection strategy without explicit supervision.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 28 of its 37 references that are in the map. First posted on arXiv 17 Jun 2026.

Builds on

28 of 37 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Sufficient Context: A New Lens on Retrieval Augmented Generation Systems9 Nov

Other references · 27

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2023Qwen Technical Reportcited by 3,273
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  5. 2017Billion-scale similarity search with GPUscited by 2,477
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  7. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  8. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  9. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  10. 2022Mass-Editing Memory in a Transformercited by 976
  11. 2024OLMo: Accelerating the Science of Language Modelscited by 599
  12. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
15 more
  1. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  2. 2024LoRA Learns Less and Forgets Lesscited by 313
  3. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  4. 2022RealTime QA: What's the Answer Right Now?cited by 239
  5. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 227
  6. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  7. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 142
  8. 2019Large Memory Layers with Product Keyscited by 120
  9. 2021Time Waits for No One! Analysis and Challenges of Temporal Misalignmentcited by 91
  10. 2024MEMORYLLM: Towards Self-Updatable Large Language Modelscited by 89
  11. 2024ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLMcited by 73
  12. 2025M+: Extending MemoryLLM with Scalable Long-Term Memorycited by 51
  13. 2024KBLaM: Knowledge Base augmented Language Modelcited by 16
  14. 2026Fast-weight Product Key Memorycited by 11
  15. 2026Knowledge Capsules: Structured Nonparametric Memory Units for LLMscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Comparing Knowledge Injection Methods for LLMs in a Low-Resource Regime
  2. 2025Unveiling Knowledge Utilization Mechanisms in LLM-based Retrieval-Augmented Generation
  3. 2024Memory3: Language Modeling with Explicit Memory
  4. 2025Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models
  5. 2025R1-Searcher++: Incentivizing the Dynamic Knowledge Acquisition of LLMs via Reinforcement Learning
  6. 2025Systematic Knowledge Injection into Large Language Models via Diverse Augmentation for Domain-Specific RAG
  7. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories
  8. 2025Resisting Contextual Interference in RAG via Parametric-Knowledge Reinforcement