mlsanityData to 14 Oct 2026

BootTOD: Bootstrap Task-oriented Dialogue Representations by Aligning Diverse Responses

arXiv 2403.01163 ↗First posted 2 Mar 2024cs.CLcited by 0

On the map: Synthetic data and self-improvement for LLMs → Code and instruction data synthesis

Abstract (arXiv)

Pre-trained language models have been successful in many scenarios. However, their usefulness in task-oriented dialogues is limited due to the intrinsic linguistic differences between general text and task-oriented dialogues. Current task-oriented dialogue pre-training methods rely on a contrastive framework, which faces challenges such as selecting true positives and hard negatives, as well as lacking diversity. In this paper, we propose a novel dialogue pre-training model called BootTOD. It learns task-oriented dialogue representations via a self-bootstrapping framework. Unlike contrastive counterparts, BootTOD aligns context and context+response representations and dismisses the requirements of contrastive pairs. BootTOD also uses multiple appropriate response targets to model the intrinsic one-to-many diversity of human conversations. Experimental results show that BootTOD outperforms strong TOD baselines on diverse downstream dialogue tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Code and instruction data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 39 of its 73 references that are in the map. First posted on arXiv 2 Mar 2024.

Builds on

39 of 73 references.

References · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  7. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  9. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  10. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
  11. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  12. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
27 more
  1. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
  2. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  3. 2019Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Datasetcited by 434
  4. 2020Understanding the Behaviour of Contrastive Losscited by 398
  5. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 386
  6. 2016A Network-based End-to-End Trainable Task-oriented Dialogue Systemcited by 252
  7. 2021PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformerscited by 204
  8. 2019TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agentscited by 188
  9. 2020TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialoguecited by 185
  10. 2020CERT: Contrastive Self-supervised Learning for Language Understandingcited by 169
  11. 2019PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variablecited by 145
  12. 2016Neural Belief Tracker: Data-Driven Dialogue State Trackingcited by 143
  13. 2019Taskmaster-1: Toward a Realistic and Diverse Dialog Datasetcited by 140
  14. 2017Key-Value Retrieval Networks for Task-Oriented Dialoguecited by 138
  15. 2018Dialogue Natural Language Inferencecited by 135
  16. 2020Boosting Contrastive Self-Supervised Learning with False Negative Cancellationcited by 119
  17. 2015Yara Parser: A Fast and Accurate Dependency Parsercited by 111
  18. 2019ConveRT: Efficient and Accurate Conversational Representations from Transformerscited by 95
  19. 2020DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialoguecited by 89
  20. 2017Frames: A Corpus for Adding Memory to Goal-Oriented Dialogue Systemscited by 85
  21. 2019Training Neural Response Selection for Task-Oriented Dialogue Systemscited by 49
  22. 2018Microsoft Dialogue Challenge: Building End-to-End Task-Completion Dialogue Systemscited by 47
  23. 2022Exploring Target Representations for Masked Autoencoderscited by 46
  24. 2022SPACE-2: Tree-Structured Semi-Supervised Contrastive Pre-training for Task-Oriented Dialog Understandingcited by 28
  25. 2021DialogueCSE: Dialogue-based Contrastive Learning of Sentence Embeddingscited by 19
  26. 2022Learning Dialogue Representations from Consecutive Utterancescited by 17
  27. 2022Semi-Supervised Knowledge-Grounded Pre-training for Task-Oriented Dialog Systemscited by 15

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Simulating Task-Oriented Dialogues with State Transition Graphs and Large Language Models
  2. 2024Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk
  3. 2024Large Language Models as Zero-shot Dialogue State Tracker through Function Calling
  4. 2023Enhancing Performance on Seen and Unseen Dialogue Scenarios using Retrieval-Augmented End-to-End Task-Oriented System
  5. 2025CoDial: Interpretable Task-Oriented Dialogue Systems Through Dialogue Flow Alignment
  6. 2024Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Training
  7. 2024Benchmarks Underestimate the Readiness of Multi-lingual Dialogue Agents
  8. 2026Context-Agent: Dynamic Discourse Trees for Non-Linear Dialogue