mlsanityData to 7 Oct 2026

DivTOD: Unleashing the Power of LLMs for Diversifying Task-Oriented Dialogue Representations

arXiv 2404.00557 ↗First posted 31 Mar 2024cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Language models pre-trained on general text have achieved impressive results in diverse fields. Yet, the distinct linguistic characteristics of task-oriented dialogues (TOD) compared to general text limit the practical utility of existing language models. Current task-oriented dialogue pre-training methods overlook the one-to-many property of conversations, where multiple responses can be appropriate given the same conversation context. In this paper, we propose a novel dialogue pre-training model called DivTOD, which collaborates with LLMs to learn diverse task-oriented dialogue representations. DivTOD guides LLMs in transferring diverse knowledge to smaller models while removing domain knowledge that contradicts task-oriented dialogues. Experiments show that our model outperforms strong TOD baselines on various downstream dialogue tasks and learns the intrinsic diversity of task-oriented dialogues.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 36 of its 46 references that are in the map. First posted on arXiv 31 Mar 2024.

Builds on

36 of 46 references.

References · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  7. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  9. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  10. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  11. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  12. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
24 more
  1. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  2. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  3. 2019Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Datasetcited by 426
  4. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 378
  5. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  6. 2022Large Language Models Are Reasoning Teacherscited by 318
  7. 2016A Network-based End-to-End Trainable Task-oriented Dialogue Systemcited by 252
  8. 2020TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialoguecited by 185
  9. 2019PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variablecited by 145
  10. 2016Neural Belief Tracker: Data-Driven Dialogue State Trackingcited by 142
  11. 2019Taskmaster-1: Toward a Realistic and Diverse Dialog Datasetcited by 139
  12. 2021Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue Systemcited by 139
  13. 2017Key-Value Retrieval Networks for Task-Oriented Dialoguecited by 137
  14. 2018Dialogue Natural Language Inferencecited by 135
  15. 2019Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Contextcited by 99
  16. 2019ConveRT: Efficient and Accurate Conversational Representations from Transformerscited by 95
  17. 2017Frames: A Corpus for Adding Memory to Goal-Oriented Dialogue Systemscited by 84
  18. 2018Microsoft Dialogue Challenge: Building End-to-End Task-Completion Dialogue Systemscited by 46
  19. 2023Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasingcited by 31
  20. 2022SPACE-2: Tree-Structured Semi-Supervised Contrastive Pre-training for Task-Oriented Dialog Understandingcited by 28
  21. 2023Let GPT be a Math Tutor: Teaching Math Word Problem Solvers with Customized Exercise Generationcited by 27
  22. 2021DialogueCSE: Dialogue-based Contrastive Learning of Sentence Embeddingscited by 19
  23. 2022Learning Dialogue Representations from Consecutive Utterancescited by 17
  24. 2022Semi-Supervised Knowledge-Grounded Pre-training for Task-Oriented Dialog Systemscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 2.

4 papers.

  1. 2024Aqulia-Med LLM: Pioneering Full-Process Open-Source Medical Language Modelscited by 4
  2. 2026CallBench: A Benchmark for Dual-Goal Coordination in Phone Call Assistantscited by 1
  3. 2025Generative AI for Critical Infrastructure in Smart Grids: A Unified Framework for Synthetic Data Generation and Anomaly Detectioncited by 0
  4. 2026Discourse Coherence and Response-Guided Context Rewriting for Multi-Party Dialogue Generationcited by 0