mlsanityData to 7 Oct 2026

Challenges in Building Intelligent Open-domain Dialog Systems

arXiv 1905.05709 ↗First posted 13 May 2019cs.CL cs.AIcited by 147

Not on the map. Why?

Abstract (arXiv)

There is a resurgent interest in developing intelligent open-domain dialog systems due to the availability of large amounts of conversational data and the recent progress on neural approaches to conversational AI. Unlike traditional task-oriented bots, an open-domain dialog system aims to establish long-term connections with users by satisfying the human need for communication, affection, and social belonging. This paper reviews the recent works on neural approaches that are devoted to addressing three challenges in developing such systems: semantics, consistency, and interactiveness. Semantics requires a dialog system to not only understand the content of the dialog but also identify user's social needs during the conversation. Consistency requires the system to demonstrate a consistent personality to win users trust and gain their long-term confidence. Interactiveness refers to the system's ability to generate interpersonal responses to achieve particular social goals such as entertainment, conforming, and task completion. The works we select to present here is based on our unique views and are by no means complete. Nevertheless, we hope that the discussion will inspire new research in developing more intelligent dialog systems.

Key facts. Cited by 147 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 36; 2022: 33; 2023: 35; 2024: 27; 2025: 12; 2026: 4.

147 papers.

Sort:

By area:

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  2. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  3. 2021Towards Emotional Support Dialog Systemscited by 294
  4. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 266
  5. 2021Internet-Augmented Dialogue Generationcited by 217
  6. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 204
  7. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
  8. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 128
  9. 2022Pretrained Language Models for Text Generation: A Surveycited by 121
  10. 2022Long Time No See! Open-Domain Conversation with Long-Term Persona Memorycited by 107
  11. 2021Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Groundingcited by 106
  12. 2024Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Codecited by 106
38 more
  1. 2023Generating Benchmarks for Factuality Evaluation of Language Modelscited by 103
  2. 2022The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systemscited by 92
  3. 2022Evaluating Human-Language Model Interactioncited by 90
  4. 2021Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approachcited by 75
  5. 2023Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Supportcited by 75
  6. 2023Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Modelscited by 67
  7. 2023A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospectscited by 66
  8. 2021Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questionscited by 63
  9. 2022Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agentscited by 62
  10. 2023Building Emotional Support Chatbots in the Era of LLMscited by 62
  11. 2024We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMscited by 61
  12. 2022Control Globally, Understand Locally: A Global-to-Local Hierarchical Graph Network for Emotional Support Conversationcited by 56
  13. 2023Generative Pre-trained Transformer: A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directionscited by 55
  14. 2023MindfulDiary: Harnessing Large Language Model to Support Psychiatric Patients' Journalingcited by 50
  15. 2024RA-ISF: Learning to Answer and Understand from Retrieval Augmentation via Iterative Self-Feedbackcited by 46
  16. 2023Watermarking Conditional Text Generation for AI Detection: Unveiling Challenges and a Semantic-Aware Watermark Remedycited by 45
  17. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 45
  18. 2022PAL: Persona-Augmented Emotional Support Conversation Generationcited by 39
  19. 2022Contrastive Learning Reduces Hallucination in Conversationscited by 39
  20. 2024UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systemscited by 38
  21. 2021NaturalConv: A Chinese Dialogue Dataset Towards Multi-turn Topic-driven Conversationcited by 35
  22. 2023ChaCha: Leveraging Large Language Models to Prompt Children to Share Their Emotions about Personal Eventscited by 31
  23. 2023DialogBench: Evaluating LLMs as Human-like Dialogue Systemscited by 31
  24. 2023LiveChat: A Large-Scale Personalized Dialogue Dataset Automatically Constructed from Live Streamingcited by 27
  25. 2021Diversifying Dialog Generation via Adaptive Label Smoothingcited by 25
  26. 2023Towards LLM-driven Dialogue State Trackingcited by 25
  27. 2024Boosting Conversational Question Answering with Fine-Grained Retrieval-Augmentation and Self-Checkcited by 24
  28. 2024SweetieChat: A Strategy-Enhanced Role-playing Framework for Diverse Scenarios Handling Emotional Support Agentcited by 24
  29. 2022"Do you follow me?": A Survey of Recent Approaches in Dialogue State Trackingcited by 22
  30. 2021Exploring Prompt-based Few-shot Learning for Grounded Dialog Generationcited by 20
  31. 2021TODSum: Task-Oriented Dialogue Summarization with State Trackingcited by 19
  32. 2024Granular Privacy Control for Geolocation with Vision Language Modelscited by 19
  33. 2025LexRAG: Benchmarking Retrieval-Augmented Generation in Multi-Turn Legal Consultation Conversationcited by 19
  34. 2022Dynamic Schema Graph Fusion Network for Multi-Domain Dialogue State Trackingcited by 18
  35. 2022CASE: Aligning Coarse-to-Fine Cognition and Affection for Empathetic Response Generationcited by 18
  36. 2022FADO: Feedback-Aware Double COntrolling Network for Emotional Support Conversationcited by 18
  37. 2024Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challengescited by 18
  38. 2023Towards Robust Personalized Dialogue Generation via Order-Insensitive Representation Regularizationcited by 17

50 most cited of 147 All 147 (JSON)