mlsanityData to 7 Oct 2026

DFM: Dialogue Foundation Model for Universal Large-Scale Dialogue-Oriented Task Learning

arXiv 2205.12662 ↗First posted 25 May 2022cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Building a universal conversational agent has been a long-standing goal of the dialogue research community. Most previous works only focus on a small set of dialogue tasks. In this work, we aim to build a unified dialogue foundation model (DFM) which can be used to solve massive diverse dialogue tasks. To achieve this goal, a large-scale well-annotated dialogue dataset with rich task diversity (DialogZoo) is collected. We introduce a framework to unify all dialogue tasks and propose novel auxiliary self-supervised tasks to achieve stable training of DFM on the highly diverse large scale DialogZoo corpus. Experiments show that, compared with models of the same size, DFM can achieve state-of-the-art or competitive performance on very rich cross-domain downstream dialogue tasks. This demonstrates that DFM largely extends the ability of unified dialogue pre-trained model.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 84 of its 103 references that are in the map. First posted on arXiv 25 May 2022.

Builds on

84 of 103 references.

References · 84

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  6. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  7. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  8. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  9. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  10. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  11. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  12. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
38 more
  1. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  2. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  3. 2018CoQA: A Conversational Question Answering Challengecited by 674
  4. 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 649
  5. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 636
  6. 2020Recipes for building an open-domain chatbotcited by 593
  7. 2017The NarrativeQA Reading Comprehension Challengecited by 590
  8. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 530
  9. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  10. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
  11. 2020Towards a Human-like Open-Domain Chatbotcited by 467
  12. 2019Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Datasetcited by 426
  13. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 419
  14. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 402
  15. 2018QuAC : Question Answering in Contextcited by 379
  16. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 378
  17. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 376
  18. 2020A Simple Language Model for Task-Oriented Dialoguecited by 331
  19. 2013Challenges in Representation Learning: A report on three machine learning contestscited by 301
  20. 2019Abductive Commonsense Reasoningcited by 300
  21. 2019Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoningcited by 281
  22. 2017The E2E Dataset: New Challenges For End-to-End Generationcited by 252
  23. 2019Massively Multilingual Neural Machine Translationcited by 246
  24. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  25. 2020Synthesizer: Rethinking Self-Attention in Transformer Modelscited by 204
  26. 2020MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselinescited by 201
  27. 2021Muppet: Massive Multi-task Representations with Pre-Finetuningcited by 192
  28. 2020TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialoguecited by 185
  29. 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 180
  30. 2019Benchmarking Natural Language Understanding Services for building Conversational Agentscited by 166
  31. 2018Neural Approaches to Conversational AIcited by 164
  32. 2021ExT5: Towards Extreme Multi-Task Scaling for Transfer Learningcited by 161
  33. 2019DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehensioncited by 156
  34. 2019PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variablecited by 145
  35. 2016Neural Belief Tracker: Data-Driven Dialogue State Trackingcited by 142
  36. 2019SParC: Cross-Domain Semantic Parsing in Contextcited by 141
  37. 2019Taskmaster-1: Toward a Realistic and Diverse Dialog Datasetcited by 139
  38. 2021Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue Systemcited by 139

50 most cited of 84 All 84 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1; 2025: 1.

3 papers.

  1. 2024MULTI: Multimodal Understanding Leaderboard with Text and Imagescited by 7
  2. 2025RetroDFM-R: Reasoning-Driven Retrosynthesis Prediction with Large Language Models via Reinforcement Learningcited by 3
  3. 2023Dialogue Agents 101: A Beginner's Guide to Critical Ingredients for Designing Effective Conversational Systemscited by 1