mlsanityData to 7 Oct 2026

Recipes for building an open-domain chatbot

arXiv 2004.13637 ↗First posted 28 Apr 2020cs.CL cs.AIcited by 593

Not on the map. Why?

Abstract (arXiv)

Building open-domain chatbots is a challenging area for machine learning research. While prior work has shown that scaling neural models in the number of parameters and the size of the data they are trained on gives improved results, we show that other ingredients are important for a high-performing chatbot. Good conversation requires a number of skills that an expert conversationalist blends in a seamless way: providing engaging talking points and listening to their partners, and displaying knowledge, empathy and personality appropriately, while maintaining a consistent persona. We show that large scale models can learn these skills when given appropriate training data and choice of generation strategy. We build variants of these recipes with 90M, 2.7B and 9.4B parameter models, and make our models and code publicly available. Human evaluations show our best models are superior to existing approaches in multi-turn dialogue in terms of engagingness and humanness measurements. We then discuss the limitations of this work by analyzing failure cases of our models.

Key facts. Cited by 593 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 113; 2022: 163; 2023: 163; 2024: 84; 2025: 53; 2026: 15.

593 papers.

Sort:

By area:

  1. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  3. 2023A Survey of Large Language Modelscited by 2,537
  4. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  5. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  6. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  7. 2023BloombergGPT: A Large Language Model for Financecited by 923
  8. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  9. 2020Aligning AI With Shared Human Valuescited by 706
  10. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  11. 2022Memory-Based Model Editing at Scalecited by 443
  12. 2021Pre-Trained Models: Past, Present and Futurecited by 319
38 more
  1. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  2. 2023Tool Learning with Foundation Modelscited by 302
  3. 2021Towards Emotional Support Dialog Systemscited by 294
  4. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
  5. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 266
  6. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 253
  7. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 238
  8. 2021Hash Layers For Large Sparse Modelscited by 236
  9. 2022OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalizationcited by 235
  10. 2021Internet-Augmented Dialogue Generationcited by 217
  11. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  12. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
  13. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  14. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 176
  15. 2022PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecastingcited by 170
  16. 2022SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualizationcited by 166
  17. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 160
  18. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  19. 2021Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answeringcited by 131
  20. 2022Generative Spoken Dialogue Language Modelingcited by 125
  21. 2022Pretrained Language Models for Text Generation: A Surveycited by 121
  22. 2024Hello Again! LLM-powered Personalized Agent for Long-term Dialoguecited by 121
  23. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 118
  24. 2023Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaborationcited by 115
  25. 2022ProsocialDialog: A Prosocial Backbone for Conversational Agentscited by 109
  26. 2022FaithDial: A Faithful Benchmark for Information-Seeking Dialoguecited by 108
  27. 2023Some things are more CRINGE than others: Iterative Preference Optimization with the Pairwise Cringe Losscited by 108
  28. 2022Long Time No See! Open-Domain Conversation with Long-Term Persona Memorycited by 107
  29. 2021Detoxifying Language Models Risks Marginalizing Minority Voicescited by 106
  30. 2022Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generationcited by 106
  31. 2021GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injectioncited by 102
  32. 2022Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completioncited by 102
  33. 2021Dialogue State Tracking with a Language Model using Schema-Driven Promptingcited by 94
  34. 2022The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systemscited by 92
  35. 2022Evaluating Human-Language Model Interactioncited by 90
  36. 2023Neurons in Large Language Models: Dead, N-gram, Positionalcited by 88
  37. 2023Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicitycited by 86
  38. 2023System 2 Attention (is something you might need too)cited by 85

50 most cited of 593 All 593 (JSON)