# AutoConv: Automatically Generating Information-seeking Conversations with Large Language Models

arXiv: https://arxiv.org/abs/2308.06507 · HTML: https://mlsanity.com/paper/2308.06507/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 42 references that are in the map.
- First posted on arXiv 12 Aug 2023.

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://mlsanity.com/paper/2209.07858/) (2022)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [UnifiedQA: Crossing Format Boundaries With a Single QA System](https://mlsanity.com/paper/2005.00700/) (2020)
- [QuAC : Question Answering in Context](https://mlsanity.com/paper/1808.07036/) (2018)
- [Towards Understanding and Mitigating Social Biases in Language Models](https://mlsanity.com/paper/2106.13219/) (2021)
- [Symbolic Knowledge Distillation: from General Language Models to Commonsense Models](https://mlsanity.com/paper/2110.07178/) (2021)
- [A Contrastive Framework for Neural Text Generation](https://mlsanity.com/paper/2202.06417/) (2022)
- [ZeroGen: Efficient Zero-shot Learning via Dataset Generation](https://mlsanity.com/paper/2202.07922/) (2022)
- [Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations](https://mlsanity.com/paper/1805.06201/) (2018)
- [Recipes for Safety in Open-domain Chatbots](https://mlsanity.com/paper/2010.07079/) (2020)
- [An Empirical Survey of Data Augmentation for Limited Data Learning in NLP](https://mlsanity.com/paper/2106.07499/) (2021)
- [Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generation](https://mlsanity.com/paper/2206.02369/) (2022)
- [On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark](https://mlsanity.com/paper/2110.08466/) (2021)
- [UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training](https://mlsanity.com/paper/2202.12359/) (2022)
- [Dialog Inpainting: Turning Documents into Dialogs](https://mlsanity.com/paper/2205.09073/) (2022)
- [DG2: Data Augmentation Through Document Grounded Dialogue Generation](https://mlsanity.com/paper/2112.08342/) (2021)

## Built on by (most cited first, 3 of 3)

- [Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systems](https://mlsanity.com/paper/2601.07072/) (2026)
- [An Energy-based Model for Word-level AutoCompletion in Computer-aided Translation](https://mlsanity.com/paper/2407.20083/) (2024)
- [Measuring Semantic Progress in Multi-turn Dialogue via Information Gain](https://mlsanity.com/paper/2606.12332/) (2026)

## Abstract (arXiv)

Information-seeking conversation, which aims to help users gather information through conversation, has achieved great progress in recent years. However, the research is still stymied by the scarcity of training data. To alleviate this problem, we propose AutoConv for synthetic conversation generation, which takes advantage of the few-shot learning ability and generation capacity of large language models (LLM). Specifically, we formulate the conversation generation problem as a language modeling task, then finetune an LLM with a few human conversations to capture the characteristics of the information-seeking process and use it for generating synthetic conversations with high quality. Experimental results on two frequently-used datasets verify that AutoConv has substantial improvements over strong baselines and alleviates the dependence on human annotation. In addition, we also provide several analysis studies to promote future research.