mlsanityData to 7 Oct 2026

Multi-Document Grounded Multi-Turn Synthetic Dialog Generation

arXiv 2409.11500 ↗First posted 17 Sep 2024cs.CL cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

We introduce a technique for multi-document grounded multi-turn synthetic dialog generation that incorporates three main ideas. First, we control the overall dialog flow using taxonomy-driven user queries that are generated with Chain-of-Thought (CoT) prompting. Second, we support the generation of multi-document grounded dialogs by mimicking real-world use of retrievers to update the grounding documents after every user-turn in the dialog. Third, we apply LLM-as-a-Judge to filter out queries with incorrect answers. Human evaluation of the synthetic dialog data suggests that the data is diverse, coherent, and includes mostly correct answers. Both human and automatic evaluations of answerable queries indicate that models fine-tuned on synthetic dialogs consistently out-perform those fine-tuned on existing human generated training data across four publicly available multi-turn document grounded benchmark test sets.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 25 of its 36 references that are in the map. First posted on arXiv 17 Sep 2024.

Builds on

25 of 36 references.

References · 25

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2018Hierarchical Neural Story Generationcited by 1,182
  10. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  11. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  12. 2018CoQA: A Conversational Question Answering Challengecited by 674
13 more
  1. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  2. 2018QuAC : Question Answering in Contextcited by 379
  3. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 150
  4. 2020Unsupervised Evaluation of Interactive Dialog with DialoGPTcited by 139
  5. 2020Open-Retrieval Conversational Question Answeringcited by 99
  6. 2023Let the LLMs Talk: Simulating Human-to-Human Conversational QA via Zero-Shot LLM-to-LLM Interactionscited by 70
  7. 2022Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agentscited by 62
  8. 2021MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documentscited by 53
  9. 2024LAB: Large-Scale Alignment for ChatBotscited by 39
  10. 2021Call for Customized Conversation: Customized Conversation Grounding Persona and Knowledgecited by 30
  11. 2020Simulated Chats for Building Dialog Systems: Learning to Generate Conversations from Instructionscited by 25
  12. 2024Structured Chain-of-Thought Prompting for Few-Shot Generation of Content-Grounded QA Conversationscited by 9
  13. 2021Summary Grounded Conversation Generationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 5.

7 papers.

  1. 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systemscited by 50
  2. 2026K-FinHallu: A Hallucination Detection Benchmark for Multi-Turn RAG in Korean Financecited by 1
  3. 2025A Library of LLM Intrinsics for Retrieval-Augmented Generationcited by 0
  4. 2026RAG-DIVE: A Dynamic Approach for Multi-Turn Dialogue Evaluation in Retrieval-Augmented Generationcited by 0
  5. 2026MTR-Suite: A Framework for Evaluating and Synthesizing Conversational Retrieval Benchmarkscited by 0
  6. 2026ConVAWG: A Retrieval-Grounded Framework for Controlled Synthetic Dialogue Generation in Violence Against Women and Girlscited by 0
  7. 2026VAKRA: Evaluating Multi-Hop Reasoning Across APIs and Retrieval Under Tool-Use Policiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Building Open-Retrieval Conversational Question Answering Systems by Generating Synthetic Data and Decontextualizing User Questions
  2. 2024DiaSynth: Synthetic Dialogue Generation Framework for Low Resource Dialogue Applications
  3. 2025Can Synthetic Query Rewrites Capture User Intent Better than Humans in Retrieval-Augmented Generation?
  4. 2024DialogAgent: An Auto-engagement Agent for Code Question Answering Data Production
  5. 2024Self-Directed Synthetic Dialogues and Revisions Technical Report
  6. 2025A Survey on Multi-Turn Interaction Capabilities of Large Language Models
  7. 2025SyGra: A Unified Graph-Based Framework for Scalable Generation, Quality Tagging, and Management of Synthetic Data
  8. 2021Retrieval-Augmented Transformer-XL for Close-Domain Dialog Generation