# Qsnail: A Questionnaire Dataset for Sequential Question Generation

arXiv: https://arxiv.org/abs/2402.14272 · HTML: https://mlsanity.com/paper/2402.14272/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 54 references that are in the map.
- First posted on arXiv 22 Feb 2024.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [RACE: Large-scale ReAding Comprehension Dataset From Examinations](https://mlsanity.com/paper/1704.04683/) (2017)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [Contrastive Decoding: Open-ended Text Generation as Optimization](https://mlsanity.com/paper/2210.15097/) (2022)
- [QuAC : Question Answering in Context](https://mlsanity.com/paper/1808.07036/) (2018)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [A Contrastive Framework for Neural Text Generation](https://mlsanity.com/paper/2202.06417/) (2022)
- [Learning to Ask: Neural Question Generation for Reading Comprehension](https://mlsanity.com/paper/1705.00106/) (2017)
- [Adaptive Information Seeking for Open-Domain Question Answering](https://mlsanity.com/paper/2109.06747/) (2021)
- [Prompting Large Language Models with Chain-of-Thought for Few-Shot Knowledge Base Question Generation](https://mlsanity.com/paper/2310.08395/) (2023)
- [Interconnected Question Generation with Coreference Alignment and Conversation Flow Modeling](https://mlsanity.com/paper/1906.06893/) (2019)
- [MacLaSa: Multi-Aspect Controllable Text Generation via Efficient Sampling from Compact Latent Space](https://mlsanity.com/paper/2305.12785/) (2023)
- [RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder for Language Modeling](https://mlsanity.com/paper/2310.10567/) (2023)
- [SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluation](https://mlsanity.com/paper/2305.11130/) (2023)
- [Topic Transferable Table Question Answering](https://mlsanity.com/paper/2109.07377/) (2021)
- [Question Generation from a Knowledge Base with Web Exploration](https://mlsanity.com/paper/1610.03807/) (2016)
- [Multi-level Adaptive Contrastive Learning for Knowledge Internalization in Dialogue Generation](https://mlsanity.com/paper/2310.08943/) (2023)
- [Difficulty Controllable Generation of Reading Comprehension Questions](https://mlsanity.com/paper/1807.03586/) (2018)

## Abstract (arXiv)

The questionnaire is a professional research methodology used for both qualitative and quantitative analysis of human opinions, preferences, attitudes, and behaviors. However, designing and evaluating questionnaires demands significant effort due to their intricate and complex structure. Questionnaires entail a series of questions that must conform to intricate constraints involving the questions, options, and overall structure. Specifically, the questions should be relevant and specific to the given research topic and intent. The options should be tailored to the questions, ensuring they are mutually exclusive, completed, and ordered sensibly. Moreover, the sequence of questions should follow a logical order, grouping similar topics together. As a result, automatically generating questionnaires presents a significant challenge and this area has received limited attention primarily due to the scarcity of high-quality datasets. To address these issues, we present Qsnail, the first dataset specifically constructed for the questionnaire generation task, which comprises 13,168 human-written questionnaires gathered from online platforms. We further conduct experiments on Qsnail, and the results reveal that retrieval models and traditional generative models do not fully align with the given research topic and intents. Large language models, while more closely related to the research topic and intents, exhibit significant limitations in terms of diversity and specificity. Despite enhancements through the chain-of-thought prompt and finetuning, questionnaires generated by language models still fall short of human-written questionnaires. Therefore, questionnaire generation is challenging and needs to be further explored. The dataset is available at: https://github.com/LeiyanGithub/qsnail.