# What should I Ask: A Knowledge-driven Approach for Follow-up Questions Generation in Conversational Surveys

arXiv: https://arxiv.org/abs/2205.10977 · HTML: https://mlsanity.com/paper/2205.10977/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 52 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation](https://mlsanity.com/paper/1603.08023/) (2016)
- [Unsupervised Evaluation of Interactive Dialog with DialoGPT](https://mlsanity.com/paper/2006.12719/) (2020)
- [Survey on Evaluation Methods for Dialogue Systems](https://mlsanity.com/paper/1905.04071/) (2019)
- [A Comprehensive Assessment of Dialog Evaluation Metrics](https://mlsanity.com/paper/2106.03706/) (2021)
- [RUBER: An Unsupervised Method for Automatic Evaluation of Open-Domain Dialog Systems](https://mlsanity.com/paper/1701.03079/) (2017)
- [Better Automatic Evaluation of Open-Domain Dialogue Systems with Contextualized Embeddings](https://mlsanity.com/paper/1904.10635/) (2019)
- [Learning an Unreferenced Metric for Online Dialogue Evaluation](https://mlsanity.com/paper/2005.00583/) (2020)
- [Learning to Ask Questions in Open-domain Conversational Systems with Typed Decoders](https://mlsanity.com/paper/1805.04843/) (2018)
- [If I Hear You Correctly: Building and Evaluating Interview Chatbots with Active Listening Skills](https://mlsanity.com/paper/2002.01862/) (2020)
- [Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questions](https://mlsanity.com/paper/1905.10700/) (2019)

## Built on by (most cited first, 13 of 13)

- [Mind the Gap Between Conversations for Improved Long-Term Dialogue Generation](https://mlsanity.com/paper/2310.15415/) (2023)
- [Follow-up Question Generation For Enhanced Patient-Provider Conversations](https://mlsanity.com/paper/2503.17509/) (2025)
- [Optimizing LLM-Based Multi-Agent System with Textual Feedback: A Case Study on Software Development](https://mlsanity.com/paper/2505.16086/) (2025)
- [Inform the uninformed: Improving Online Informed Consent Reading with an AI-Powered Chatbot](https://mlsanity.com/paper/2302.00832/) (2023)
- [Telephone Surveys Meet Conversational AI: Evaluating a LLM-Based Telephone Survey System at Scale](https://mlsanity.com/paper/2502.20140/) (2025)
- [Feedback by Design: Understanding and Overcoming User Feedback Barriers in Conversational Agents](https://mlsanity.com/paper/2602.01405/) (2026)
- [Making Task-Oriented Dialogue Datasets More Natural by Synthetically Generating Indirect User Requests](https://mlsanity.com/paper/2406.07794/) (2024)
- [A Career Interview Dialogue System using Large Language Model-based Dynamic Slot Generation](https://mlsanity.com/paper/2412.16943/) (2024)
- [Bridging Information Gaps with Comprehensive Answers: Improving the Diversity and Informativeness of Follow-Up Questions](https://mlsanity.com/paper/2502.17715/) (2025)
- [From Superficial to Deep: Integrating External Knowledge for Follow-up Question Generation Using Knowledge Graph and LLM](https://mlsanity.com/paper/2504.05801/) (2025)
- [Learning to Retrieve Engaging Follow-Up Queries](https://mlsanity.com/paper/2302.10978/) (2023)
- [Follow-Up Questions Improve Documents Generated by Large Language Models](https://mlsanity.com/paper/2407.12017/) (2024)
- [FeedAIde: Guiding App Users to Submit Rich Feedback Reports by Asking Context-Aware Follow-Up Questions](https://mlsanity.com/paper/2603.04244/) (2026)

## Abstract (arXiv)

Generating follow-up questions on the fly could significantly improve conversational survey quality and user experiences by enabling a more dynamic and personalized survey structure. In this paper, we proposed a novel task for knowledge-driven follow-up question generation in conversational surveys. We constructed a new human-annotated dataset of human-written follow-up questions with dialogue history and labeled knowledge in the context of conversational surveys. Along with the dataset, we designed and validated a set of reference-free Gricean-inspired evaluation metrics to systematically evaluate the quality of generated follow-up questions. We then propose a two-staged knowledge-driven model for the task, which generates informative and coherent follow-up questions by using knowledge to steer the generation process. The experiments demonstrate that compared to GPT-based baseline models, our two-staged model generates more informative, coherent, and clear follow-up questions.