mlsanityData to 7 Oct 2026

Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk

arXiv 2401.05033 ↗First posted 10 Jan 2024cs.CL cs.AIcited by 34

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

also in 1 more
Abstract (arXiv)

Large language models (LLMs) are powerful dialogue agents, but specializing them towards fulfilling a specific function can be challenging. Instructing tuning, i.e. tuning models on instruction and sample responses generated by humans (Ouyang et al., 2022), has proven as an effective method to do so, yet requires a number of data samples that a) might not be available or b) costly to generate. Furthermore, this cost increases when the goal is to make the LLM follow a specific workflow within a dialogue instead of single instructions. Inspired by the self-play technique in reinforcement learning and the use of LLMs to simulate human agents, we propose a more effective method for data collection through LLMs engaging in a conversation in various roles. This approach generates a training data via "self-talk" of LLMs that can be refined and utilized for supervised fine-tuning. We introduce an automated way to measure the (partial) success of a dialogue. This metric is used to filter the generated conversational data that is fed back in LLM for training. Based on our automated and human evaluations of conversation quality, we demonstrate that such self-talk data improves results. In addition, we examine the various characteristics that showcase the quality of generated dialogues and how they can be connected to their potential utility as training data.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 32 of its 52 references that are in the map. First posted on arXiv 10 Jan 2024.

Builds on

32 of 52 references.

References · 32

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  7. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  8. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  9. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  10. 2018Hierarchical Neural Story Generationcited by 1,182
  11. 2023Teaching Large Language Models to Self-Debugcited by 963
  12. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
20 more
  1. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  2. 2022Self-critiquing models for assisting human evaluatorscited by 372
  3. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 244
  4. 2023Self-Consuming Generative Models Go MADcited by 239
  5. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  6. 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 203
  7. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 185
  8. 2023Training Language Models with Language Feedback at Scalecited by 125
  9. 2019Learning to Speak and Act in a Fantasy Text Adventure Gamecited by 122
  10. 2018Learning to Write with Cooperative Discriminatorscited by 119
  11. 2021A Survey on Gender Bias in Natural Language Processingcited by 89
  12. 2021Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systemscited by 73
  13. 2018Building a Conversational Agent Overnight with Dialogue Self-Playcited by 70
  14. 2023Self-Verification Improves Few-Shot Clinical Information Extractioncited by 52
  15. 2022Controllable Dialogue Simulation with In-Context Learningcited by 42
  16. 2023SGP-TOD: Building Task Bots Effortlessly via Schema-Guided LLM Promptingcited by 39
  17. 2021Disembodied Machine Learning: On the Illusion of Objectivity in NLPcited by 37
  18. 2017An optimal transportation approach for assessing almost stochastic ordercited by 33
  19. 2020Deploying Lifelong Open-Domain Dialogue Learningcited by 17
  20. 2022A Generative User Simulator with GPT-based Architecture and Goal State Tracking for Reinforced Multi-Domain Dialog Systemscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 21; 2025: 8; 2026: 5.

34 papers.

Sort:

By area:

  1. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
  2. 2024Systematic Biases in LLM Simulations of Debatescited by 98
  3. 2024A Survey on Self-Evolution of Large Language Modelscited by 86
  4. 2024Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinementcited by 86
  5. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  6. 2024Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMscited by 59
  7. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 45
  8. 2024Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verificationcited by 43
  9. 2024EvoChart: A Benchmark and a Self-Training Approach Towards Real-World Chart Understandingcited by 32
  10. 2024Enhancing Dialogue State Tracking Models through LLM-backed User-Agents Simulationcited by 24
  11. 2024How to Synthesize Text Data without Model Collapse?cited by 24
  12. 2025MCPXKIT: The Unified Toolkit for Analyzing Model Context Protocol Securitycited by 21
22 more
  1. 2024GLEE: A Unified Framework and Benchmark for Language-based Economic Environmentscited by 19
  2. 2024Can LLMs Replace Economic Choice Prediction Labs? The Case of Language-based Persuasion Gamescited by 12
  3. 2025Evaluation and Facilitation of Online Discussions in the LLM Era: A Surveycited by 8
  4. 2025Why Less is More (Sometimes): A Theory of Data Curationcited by 8
  5. 2024Simulating Task-Oriented Dialogues with State Transition Graphs and Large Language Modelscited by 6
  6. 2024AgentRE: An Agent-Based Framework for Navigating Complex Information Landscapes in Relation Extractioncited by 6
  7. 2024Stephanie: Step-by-Step Dialogues for Mimicking Human Interactions in Social Conversationscited by 5
  8. 2024Uncertainty-Guided Self-Questioning and Answering for Video-Language Alignmentcited by 4
  9. 2025Learning Together to Perform Better: Teaching Small-Scale LLMs to Collaborate via Preferential Rationale Tuningcited by 3
  10. 2025Expectation Confirmation Preference Optimization for Multi-Turn Conversational Recommendation Agentcited by 3
  11. 2026TurnWise: The Gap between Single- and Multi-turn Language Model Capabilitiescited by 3
  12. 2024Controllable and Reliable Knowledge-Intensive Task-Oriented Conversational Agents with Declarative Genie Worksheetscited by 2
  13. 2024What should I wear to a party in a Greek taverna? Evaluation for Conversational Agents in the Fashion Domaincited by 2
  14. 2024LLMs are Superior Feedback Providers: Bootstrapping Reasoning for Lie Detection with Self-Generated Feedbackcited by 2
  15. 2025Designing Synthetic Discussion Generation Systems: A Case Study for Online Facilitationcited by 2
  16. 2024MentalArena: Self-play Training of Language Models for Diagnosis and Treatment of Mental Health Disorderscited by 1
  17. 2024METEOR: Evolutionary Journey of Large Language Models from Guidance to Self-Growthcited by 1
  18. 2025It Helps to Take a Second Opinion: Teaching Smaller LLMs to Deliberate Mutually via Selective Rationale Optimisationcited by 1
  19. 2026Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challengescited by 1
  20. 2026Maximizing Mutual Information Between Prompt and Response Improves LLM Performance With No Additional Datacited by 0
  21. 2026LLM Agents for Deliberative Collaboration: A Study on Joint Decision Making Under Partial Observabilitycited by 0
  22. 2026An Auditable Symbolic-RAG-Generative AI Architecture for Goal-Oriented Conversation Orchestrationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Interactive Dialogue Agents via Reinforcement Learning on Hindsight Regenerations
  2. 2024Sparse Rewards Can Self-Train Dialogue Agents
  3. 2024Evaluating Very Long-Term Conversational Memory of LLM Agents
  4. 2024Should We Fine-Tune or RAG? Evaluating Different Techniques to Adapt LLMs for Dialogue
  5. 2023Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversations
  6. 2024Rewriting Conversational Utterances with Instructed Large Language Models
  7. 2025Self-Improving LLM Agents at Test-Time
  8. 2025Training LLM-Based Agents with Synthetic Self-Reflected Trajectories and Partial Masking