# A Survey of Available Corpora for Building Data-Driven Dialogue Systems

arXiv: https://arxiv.org/abs/1512.05742 · HTML: https://mlsanity.com/paper/1512.05742/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2015.

## Built on by (most cited first, 30 of 33)

- [BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage](https://mlsanity.com/paper/2208.03188/) (2022)
- [QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension](https://mlsanity.com/paper/2107.12708/) (2021)
- [A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4](https://mlsanity.com/paper/2310.12321/) (2023)
- [Multi-Party Chat: Conversational Agents in Group Settings with Humans and Models](https://mlsanity.com/paper/2304.13835/) (2023)
- [The CRINGE Loss: Learning what language not to model](https://mlsanity.com/paper/2211.05826/) (2022)
- [Survey on reinforcement learning for language processing](https://mlsanity.com/paper/2104.05565/) (2021)
- [Rethinking and Refining the Distinct Metric](https://mlsanity.com/paper/2202.13587/) (2022)
- [Why is constrained neural language generation particularly challenging?](https://mlsanity.com/paper/2206.05395/) (2022)
- [A Survey on Deep Reinforcement Learning for Audio-Based Applications](https://mlsanity.com/paper/2101.00240/) (2021)
- [A Survey of Intent Classification and Slot-Filling Datasets for Task-Oriented Dialog](https://mlsanity.com/paper/2207.13211/) (2022)
- [Farmer.Chat: Scaling AI-Powered Agricultural Services for Smallholder Farmers](https://mlsanity.com/paper/2409.08916/) (2024)
- [Learning from data in the mixed adversarial non-adversarial case: Finding the helpers and ignoring the trolls](https://mlsanity.com/paper/2208.03295/) (2022)
- [Improving Open Language Models by Learning from Organic Interactions](https://mlsanity.com/paper/2306.04707/) (2023)
- [A System for Human-AI collaboration for Online Customer Support](https://mlsanity.com/paper/2301.12158/) (2023)
- [The Case for Developing a Foundation Model for Planning-like Tasks from Scratch](https://mlsanity.com/paper/2404.04540/) (2024)
- [Characterizing Language Use in a Collaborative Situated Game](https://mlsanity.com/paper/2512.03381/) (2025)
- [When is it permissible for artificial intelligence to lie? A trust-based approach](https://mlsanity.com/paper/2103.05434/) (2021)
- [Quick Starting Dialog Systems with Paraphrase Generation](https://mlsanity.com/paper/2204.02546/) (2022)
- [End-to-End Evaluation of a Spoken Dialogue System for Learning Basic Mathematics](https://mlsanity.com/paper/2211.03511/) (2022)
- [Data Augmentation for Modeling Human Personality: The Dexter Machine](https://mlsanity.com/paper/2301.08606/) (2023)
- [Advising Agent for Service-Providing Live-Chat Operators](https://mlsanity.com/paper/2105.03986/) (2021)
- [Building a Legal Dialogue System: Development Process, Challenges and Opportunities](https://mlsanity.com/paper/2109.00381/) (2021)
- [A Spoken Drug Prescription Dataset in French for Spoken Language Understanding](https://mlsanity.com/paper/2207.08292/) (2022)
- [Talking with Machines: A Comprehensive Survey of Emergent Dialogue Systems](https://mlsanity.com/paper/2305.16324/) (2023)
- [Unstructured Text Enhanced Open-domain Dialogue System: A Systematic Survey](https://mlsanity.com/paper/2411.09166/) (2024)
- [HumAIne-Chatbot: Real-Time Personalized Conversational AI via Reinforcement Learning](https://mlsanity.com/paper/2509.04303/) (2025)
- [A survey of joint intent detection and slot-filling models in natural language understanding](https://mlsanity.com/paper/2101.08091/) (2021)
- [A Bi-Encoder LSTM Model For Learning Unstructured Dialogs](https://mlsanity.com/paper/2104.12269/) (2021)
- [A Brief Study on the Effects of Training Generative Dialogue Models with a Semantic loss](https://mlsanity.com/paper/2106.10619/) (2021)
- [Inspecting Spoken Language Understanding from Kids for Basic Math Learning at Home](https://mlsanity.com/paper/2306.00482/) (2023)

## Abstract (arXiv)

During the past decade, several areas of speech and language understanding have witnessed substantial breakthroughs from the use of data-driven models. In the area of dialogue systems, the trend is less obvious, and most practical systems are still built through significant engineering and expert knowledge. Nevertheless, several recent results suggest that data-driven approaches are feasible and quite promising. To facilitate research in this area, we have carried out a wide survey of publicly available datasets suitable for data-driven learning of dialogue systems. We discuss important characteristics of these datasets, how they can be used to learn diverse dialogue strategies, and their other potential uses. We also examine methods for transfer learning between datasets and the use of external knowledge. Finally, we discuss appropriate choice of evaluation metrics for the learning objective.