# Structured Chain-of-Thought Prompting for Few-Shot Generation of Content-Grounded QA Conversations

arXiv: https://arxiv.org/abs/2402.11770 · HTML: https://mlsanity.com/paper/2402.11770/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 24 of its 24 references that are in the map.
- First posted on arXiv 19 Feb 2024.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 24 of 24)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Show Your Work: Scratchpads for Intermediate Computation with Language Models](https://mlsanity.com/paper/2112.00114/) (2021)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [QuAC : Question Answering in Context](https://mlsanity.com/paper/1808.07036/) (2018)
- [UL2: Unifying Language Learning Paradigms](https://mlsanity.com/paper/2205.05131/) (2022)
- [ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models](https://mlsanity.com/paper/2305.18323/) (2023)
- [TopiOCQA: Open-domain Conversational Question Answering with Topic Switching](https://mlsanity.com/paper/2110.00768/) (2021)
- [Detecting and Mitigating Hallucinations in Multilingual Summarisation](https://mlsanity.com/paper/2305.13632/) (2023)
- [Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection](https://mlsanity.com/paper/2301.07779/) (2023)
- [DelucionQA: Detecting Hallucinations in Domain-specific Question Answering](https://mlsanity.com/paper/2312.05200/) (2023)
- [DoQA -- Accessing Domain-Specific FAQs via Conversational QA](https://mlsanity.com/paper/2005.01328/) (2020)
- [Reason first, then respond: Modular Generation for Knowledge-infused Dialogue](https://mlsanity.com/paper/2111.05204/) (2021)
- [Generative Knowledge Selection for Knowledge-Grounded Dialogues](https://mlsanity.com/paper/2304.04836/) (2023)

## Built on by (most cited first, 9 of 9)

- [Deep Research Agents: A Systematic Examination And Roadmap](https://mlsanity.com/paper/2506.18096/) (2025)
- [Large Language Models Hallucination: A Comprehensive Survey](https://mlsanity.com/paper/2510.06265/) (2025)
- [Multi-Document Grounded Multi-Turn Synthetic Dialog Generation](https://mlsanity.com/paper/2409.11500/) (2024)
- [Characterizing, Evaluating, and Optimizing Complex Reasoning](https://mlsanity.com/paper/2602.08498/) (2026)
- [Redefining Proactivity for Information Seeking Dialogue](https://mlsanity.com/paper/2410.15297/) (2024)
- [SRLCG: Self-Rectified Large-Scale Code Generation with Multidimensional Chain-of-Thought and Dynamic Backtracking](https://mlsanity.com/paper/2504.00532/) (2025)
- [Diversity Augmentation of Dynamic User Preference Data for Boosting Personalized Text Summarizers](https://mlsanity.com/paper/2510.10082/) (2025)
- [Generating Place-Based Compromises Between Two Points of View](https://mlsanity.com/paper/2604.24536/) (2026)
- [A Hybrid LLM-Based Framework for Automated Security Annotation Generation in Business Process Models](https://mlsanity.com/paper/2608.14370/) (2026)

## Abstract (arXiv)

We introduce a structured chain-of-thought (SCoT) prompting approach to generating content-grounded multi-turn question-answer conversations using a pre-trained large language model (LLM). At the core of our proposal is a structured breakdown of the complex task into a number of states in a state machine, so that actions corresponding to various subtasks, e.g., content reading and utterance generation, can be executed in their own dedicated states. Each state leverages a unique set of resources including prompts and (optionally) additional tools to augment the generation process. Our experimental results show that SCoT prompting with designated states for hallucination mitigation increases agent faithfulness to grounding documents by up to 16.8%. When used as training data, our open-domain conversations synthesized from only 6 Wikipedia-based seed demonstrations train strong conversational QA agents; in out-of-domain evaluation, for example, we observe improvements of up to 13.9% over target domain gold data when the latter is augmented with our generated examples.