mlsanityData to 14 Oct 2026

Augmenting Transformers with KNN-Based Composite Memory for Dialogue

arXiv 2004.12744 ↗First posted 27 Apr 2020cs.CLcited by 53

Not on the map. Why?

Abstract (arXiv)

Various machine learning tasks can benefit from access to external information of different modalities, such as text and images. Recent work has focused on learning architectures with large memories capable of storing this knowledge. We propose augmenting generative Transformer neural networks with KNN-based Information Fetching (KIF) modules. Each KIF module learns a read operation to access fixed external knowledge. We apply these modules to generative dialog modeling, a challenging task where information must be flexibly retrieved and incorporated to maintain the topic and flow of conversation. We demonstrate the effectiveness of our approach by identifying relevant knowledge required for knowledgeable but engaging dialog from Wikipedia, images, and human-written dialog utterances, and show that leveraging this retrieved information improves model performance, measured by automatic and human evaluation.

Key facts. Cited by 53 papers in the map (data through 14 Oct 2026). First posted on arXiv 27 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 13; 2022: 18; 2023: 10; 2024: 8; 2025: 2; 2026: 2.

53 papers.

Sort:

By area:

  1. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
  2. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 354
  3. 2022Memorizing Transformerscited by 226
  4. 2021End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answeringcited by 133
  5. 2022Training Language Models with Memory Augmentationcited by 113
  6. 2023Complex Claim Verification with Evidence Retrieved in the Wildcited by 110
  7. 2023Retrieving Multimodal Information for Augmented Generation: A Surveycited by 85
  8. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  9. 2024Retrieval Augmented Time Series Forecastingcited by 46
  10. 2021Fast Nearest Neighbor Machine Translationcited by 37
  11. 2022An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Taskscited by 36
  12. 2021Knowledge Enhanced Pretrained Language Models: A Compreshensive Surveycited by 33
38 more
  1. 2021GNN-LM: Language Modeling based on Global Contexts via GNNcited by 27
  2. 2022Relational Memory Augmented Language Modelscited by 24
  3. 2022Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Modelscited by 24
  4. 2022Chunk-based Nearest Neighbor Machine Translationcited by 21
  5. 2021Cross-Modal Retrieval Augmentation for Multi-Modal Classificationcited by 20
  6. 2022You can't pick your neighbors, or can you? When and how to rely on retrieval in the kNN-LMcited by 18
  7. 2021Learning to Select External Knowledge with Multi-Scale Negative Samplingcited by 16
  8. 2023Rethinking Privacy in Machine Learning Pipelines from an Information Flow Control Perspectivecited by 16
  9. 2021RETRONLU: Retrieval Augmented Task-Oriented Semantic Parsingcited by 15
  10. 2024DR-RAG: Applying Dynamic Document Relevance to Retrieval-Augmented Generation for Question-Answeringcited by 14
  11. 2023Simple and Scalable Nearest Neighbor Machine Translationcited by 12
  12. 2024Retrieval-Enhanced Machine Learning: Synthesis and Opportunitiescited by 11
  13. 2022Incorporating Explicit Knowledge in Pre-trained Language Models for Passage Re-rankingcited by 10
  14. 2021Faster Nearest Neighbor Machine Translationcited by 8
  15. 2021∞-former: Infinite Memory Transformercited by 7
  16. 2023MOKA: Moral Knowledge Augmentation for Moral Event Extractioncited by 7
  17. 2022kNN-NER: Named Entity Recognition with Nearest Neighbor Searchcited by 6
  18. 2022Injecting Domain Knowledge in Language Models for Task-Oriented Dialogue Systemscited by 6
  19. 2022Long-term Control for Dialogue Generation: Methods and Evaluationcited by 5
  20. 2022Knowledge-grounded Dialog State Trackingcited by 5
  21. 2022Linearizing Transformer with Key-Value Memorycited by 4
  22. 2023Hybrid-RACA: Hybrid Retrieval-Augmented Composition Assistance for Real-time Text Predictioncited by 4
  23. 2021A Neighbourhood Framework for Resource-Lean Content Flaggingcited by 3
  24. 2021Generating Related Workcited by 3
  25. 2021Understanding and Improving the Exemplar-based Generation for Open-domain Conversationcited by 3
  26. 2022GNN-SL: Sequence Labeling Based on Nearest Examples via GNNcited by 3
  27. 2024RoboMP2: A Robotic Multimodal Perception-Planning Framework with Multimodal Large Language Modelscited by 3
  28. 2026Controlling Long-Horizon Behavior in Language Model Agents with Explicit State Dynamicscited by 3
  29. 2025When More Retrieval Hurts: Retrieval-Augmented Code Review Generationcited by 2
  30. 2022Memory in humans and deep language models: Linking hypotheses for model augmentationcited by 1
  31. 2022Mind the Knowledge Gap: A Survey of Knowledge-enhanced Dialogue Systemscited by 1
  32. 2023Nearest Neighbor Machine Translation is Meta-Optimizer on Output Projection Layercited by 1
  33. 2023Neural Conversation Models and How to Rein Them in: A Survey of Failures and Fixescited by 1
  34. 2023DRAFT: Dense Retrieval Augmented Few-shot Topic classifier Frameworkcited by 1
  35. 2024Extended Mind Transformerscited by 1
  36. 2024Unstructured Text Enhanced Open-domain Dialogue System: A Systematic Surveycited by 1
  37. 2022Transferring Knowledge via Neighborhood-Aware Optimal Transport for Low-Resource Hate Speech Detectioncited by 0
  38. 2023Heterogenous Memory Augmented Neural Networkscited by 0

50 most cited of 53 All 53 (JSON)