mlsanityData to 14 Oct 2026

Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion

arXiv 2203.13224 ↗First posted 24 Mar 2022cs.CL cs.AIcited by 102

Not on the map. Why?

Abstract (arXiv)

Language models (LMs) have recently been shown to generate more factual responses by employing modularity (Zhou et al., 2021) in combination with retrieval (Adolphs et al., 2021). We extend the recent approach of Adolphs et al. (2021) to include internet search as a module. Our SeeKeR (Search engine->Knowledge->Response) method thus applies a single LM to three modular tasks in succession: search, generating knowledge, and generating a final response. We show that, when using SeeKeR as a dialogue model, it outperforms the state-of-the-art model BlenderBot 2 (Chen et al., 2021) on open-domain knowledge-grounded conversations for the same number of parameters, in terms of consistency, knowledge and per-turn engagingness. SeeKeR applied to topical prompt completions as a standard language model outperforms GPT2 (Radford et al., 2019) and GPT3 (Brown et al., 2020) in terms of factuality and topicality, despite GPT3 being a vastly larger model. Our code and models are made publicly available.

Key facts. Cited by 102 papers in the map (data through 14 Oct 2026). Builds on 47 of its 55 references that are in the map. First posted on arXiv 24 Mar 2022.

Builds on

47 of 55 references.

References · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  8. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  9. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  10. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  11. 2016Pointer Sentinel Mixture Modelscited by 3,096
  12. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
35 more
  1. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  2. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  3. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  4. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  5. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  6. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  7. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 828
  8. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  9. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  10. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
  11. 2020Recipes for building an open-domain chatbotcited by 593
  12. 2021Editing Factual Knowledge in Language Modelscited by 553
  13. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 548
  14. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  15. 2021Fast Model Editing at Scalecited by 498
  16. 2020Towards a Human-like Open-Domain Chatbotcited by 470
  17. 2021BASE Layers: Simplifying Training of Large, Sparse Modelscited by 320
  18. 2020The Pushshift Reddit Datasetcited by 292
  19. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 267
  20. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  21. 2021Internet-Augmented Dialogue Generationcited by 217
  22. 2020Nearest Neighbor Machine Translationcited by 209
  23. 2020Recipes for Safety in Open-domain Chatbotscited by 170
  24. 2019Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoringcited by 165
  25. 2019Queens are Powerful too: Mitigating Gender Bias in Dialogue Generationcited by 152
  26. 2020Can You Put it All Together: Evaluating Conversational Agents' Ability to Blend Skillscited by 131
  27. 2018Training Millions of Personalized Dialogue Agentscited by 126
  28. 2019Learning to Speak and Act in a Fantasy Text Adventure Gamecited by 122
  29. 2019ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisonscited by 83
  30. 2021Adaptive Semiparametric Language Modelscited by 83
  31. 2016Improving Neural Language Models with a Continuous Cachecited by 82
  32. 2019The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agentscited by 45
  33. 2018Learning Semantic Textual Similarity from Conversationscited by 41
  34. 2021Reason first, then respond: Modular Generation for Knowledge-infused Dialoguecited by 32
  35. 2021Think Before You Speak: Explicitly Generating Implicit Commonsense Knowledge for Response Generationcited by 26

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 22; 2023: 49; 2024: 18; 2025: 10; 2026: 3.

102 papers.

Sort:

By area:

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  3. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  4. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  5. 2023Enabling Large Language Models to Generate Text with Citationscited by 595
  6. 2022Improving alignment of dialogue agents via targeted human judgementscited by 531
  7. 2023Augmented Language Models: a Surveycited by 403
  8. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  9. 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 213
  10. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 198
  11. 2023ART: Automatic multi-step reasoning and tool-use for large language modelscited by 197
  12. 2023Personality Traits in Large Language Modelscited by 169
38 more
  1. 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 154
  2. 2022Rethinking with Retrieval: Faithful Large Language Model Inferencecited by 145
  3. 2022Generative Spoken Dialogue Language Modelingcited by 126
  4. 2023Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaborationcited by 116
  5. 2022Autoregressive Search Engines: Generating Substrings as Document Identifierscited by 113
  6. 2023A Survey of Text Watermarking in the Era of Large Language Modelscited by 112
  7. 2023MemoChat: Tuning LLMs to Use Memos for Consistent Long-Range Open-Domain Conversationcited by 97
  8. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 93
  9. 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 88
  10. 2022A Survey of Knowledge Enhanced Pre-trained Language Modelscited by 76
  11. 2023WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipediacited by 67
  12. 2023Multi-Party Chat: Conversational Agents in Group Settings with Humans and Modelscited by 54
  13. 2024Conformal Alignment: Knowing When to Trust Foundation Models with Guaranteescited by 54
  14. 2023Misusing Tools in Large Language Models With Visual Adversarial Examplescited by 42
  15. 2023Legal Syllogism Prompting: Teaching Large Language Models for Legal Judgment Predictioncited by 38
  16. 2022Learning New Skills after Deployment: Improving open-domain internet-driven dialogue with human feedbackcited by 36
  17. 2023Learn to Refuse: Making Large Language Models More Controllable and Reliable through Knowledge Scope Limitation and Refusal Mechanismcited by 33
  18. 2023Controllable Mixed-Initiative Dialogue Generation through Promptingcited by 32
  19. 2023Training Models to Generate, Recognize, and Reframe Unhelpful Thoughtscited by 31
  20. 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directionscited by 31
  21. 2023NevIR: Negation in Neural Information Retrievalcited by 29
  22. 2023Information Retrieval Meets Large Language Models: A Strategic Report from Chinese IR Communitycited by 29
  23. 2024KnowHalu: Hallucination Detection via Multi-Form Knowledge Based Factual Checkingcited by 26
  24. 2023Chameleon: a Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Modelscited by 23
  25. 2022Reflect, Not Reflex: Inference-Based Common Ground Improves Dialogue Response Qualitycited by 22
  26. 2023Transformer models: an introduction and catalogcited by 21
  27. 2023Effective Theory of Transformers at Initializationcited by 21
  28. 2022When Life Gives You Lemons, Make Cherryade: Converting Feedback from Bad Responses into Good Labelscited by 20
  29. 2023Augmenting LLMs with Knowledge: A survey on hallucination preventioncited by 16
  30. 2024The "Colonial Impulse" of Natural Language Processing: An Audit of Bengali Sentiment Analysis Tools and Their Identity-based Biasescited by 16
  31. 2023Quantifying and Attributing the Hallucination of Large Language Models via Association Analysiscited by 15
  32. 2024Social Skill Training with Large Language Modelscited by 15
  33. 2023ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Humancited by 14
  34. 2023Leveraging Implicit Feedback from Deployment Data in Dialoguecited by 14
  35. 2023Fundamentals of Generative Large Language Models and Perspectives in Cyber-Defensecited by 13
  36. 2024Are foundation models for computer vision good conformal predictors?cited by 13
  37. 2023Large Language Model Programscited by 11
  38. 2022Q-TOD: A Query-driven Task-oriented Dialogue Systemcited by 10

50 most cited of 102 All 102 (JSON)