mlsanityData to 7 Oct 2026

Large Language Models with Controllable Working Memory

arXiv 2211.05110 ↗First posted 9 Nov 2022cs.CL cs.AI cs.LGcited by 169

Not on the map. Why?

Cited by 38 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP), owing to their excellent understanding and generation abilities. Remarkably, what further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. While many downstream applications provide the model with an informational context to aid its performance on the underlying task, how the model's world knowledge interacts with the factual information presented in the context remains under explored. As a desirable behavior, an LLM should give precedence to the context whenever it contains task-relevant information that conflicts with the model's memorized knowledge. This enables model predictions to be grounded in the context, which can then be used to update or correct specific model predictions without frequent retraining. By contrast, when the context is irrelevant to the task, the model should ignore it and fall back on its internal knowledge. In this paper, we undertake a first joint study of the aforementioned two properties, namely controllability and robustness, in the context of LLMs. We demonstrate that state-of-the-art T5 and PaLM (both pretrained and finetuned) could exhibit poor controllability and robustness, which do not scale with increasing model size. As a solution, we propose a novel method - Knowledge Aware FineTuning (KAFT) - to strengthen both controllability and robustness by incorporating counterfactual and irrelevant contexts to standard supervised datasets. Our comprehensive evaluation showcases the utility of KAFT across model architectures and sizes.

Key facts. Cited by 169 papers in the map (data through 7 Oct 2026). Builds on 30 of its 51 references that are in the map. First posted on arXiv 9 Nov 2022.

Builds on

30 of 51 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2022Classifier-Free Diffusion Guidancecited by 5,717
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  8. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  9. 2019Language Models as Knowledge Bases?cited by 1,922
  10. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  11. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  12. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
18 more
  1. 2022Mass-Editing Memory in a Transformercited by 976
  2. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  3. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  4. 2021Editing Factual Knowledge in Language Modelscited by 548
  5. 2021Fast Model Editing at Scalecited by 491
  6. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  7. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
  8. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 290
  9. 2019QASC: A Dataset for Question Answering via Sentence Compositioncited by 248
  10. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 240
  11. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  12. 2020Modifying Memories in Transformer Modelscited by 222
  13. 2020How Context Affects Language Models' Factual Predictionscited by 181
  14. 2021Towards Continual Knowledge Learning of Language Modelscited by 172
  15. 2022UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Trainingcited by 57
  16. 2022Entity Cloze By Date: What LMs Know About Unseen Entitiescited by 55
  17. 2022A Causal Lens for Controllable Text Generationcited by 45
  18. 2020Contextualized Representations Using Textual Encyclopedic Knowledgecited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 34; 2024: 81; 2025: 36; 2026: 17.

169 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  2. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  3. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  4. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  5. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 329
  6. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 307
  7. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  8. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  9. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  10. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  11. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  12. 2023RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Modelscited by 173
38 more
  1. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  2. 2024To Believe or Not to Believe Your LLMcited by 120
  3. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  4. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 119
  5. 2024Empowering Biomedical Discovery with AI Agentscited by 117
  6. 2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 112
  7. 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 107
  8. 2023R-Tuning: Instructing Large Language Models to Say `I Don't Know'cited by 106
  9. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  10. 2022DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answeringcited by 93
  11. 2023Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrieverscited by 91
  12. 2024Large Language Models and Games: A Survey and Roadmapcited by 91
  13. 2024Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Trainingcited by 88
  14. 2025HalluLens: LLM Hallucination Benchmarkcited by 88
  15. 2024Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Modelscited by 87
  16. 2024BadEdit: Backdooring large language models by model editingcited by 84
  17. 2023Context-faithful Prompting for Large Language Modelscited by 76
  18. 2024Learning to Route LLMs with Confidence Tokenscited by 66
  19. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  20. 2025Evaluating LLM-based Agents for Multi-Turn Conversations: A Surveycited by 56
  21. 2023Resolving Knowledge Conflicts in Large Language Modelscited by 49
  22. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 45
  23. 2024Understanding the Dark Side of LLMs' Intrinsic Self-Correctioncited by 42
  24. 2023Merging Generated and Retrieved Knowledge for Open-Domain QAcited by 36
  25. 2024IRCAN: Mitigating Knowledge Conflicts in LLM Generation via Identifying and Reweighting Context-Aware Neuronscited by 36
  26. 2023How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advancescited by 33
  27. 2024In-Context Editing: Learning Knowledge from Self-Induced Distributionscited by 33
  28. 2025The Distracting Effect: Understanding Irrelevant Passages in RAGcited by 32
  29. 2023An In-depth Survey of Large Language Model-based Artificial Intelligence Agentscited by 30
  30. 2024Grounding Language Model with Chunking-Free In-Context Retrievalcited by 30
  31. 2024Source-Aware Training Enables Knowledge Attribution in Language Modelscited by 29
  32. 2023Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applicationscited by 28
  33. 2024ICT: Image-Object Cross-Level Trusted Intervention for Mitigating Object Hallucination in Large Vision-Language Modelscited by 28
  34. 2025Temporal Working Memory: Query-Guided Segment Refinement for Enhanced Multimodal Understandingcited by 28
  35. 2024Rowen: Adaptive Retrieval-Augmented Generation for Hallucination Mitigation in LLMscited by 27
  36. 2024Discerning and Resolving Knowledge Conflicts through Adaptive Decoding with Contextual Information-Entropy Constraintcited by 27
  37. 2025FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generationcited by 27
  38. 2024Cofca: A Step-Wise Counterfactual Multi-hop QA benchmarkcited by 26

50 most cited of 169 All 169 (JSON)