# AI-Assisted Conversational Interviewing: Effects on Data Quality and Respondent Experience

arXiv: https://arxiv.org/abs/2504.13908 · HTML: https://mlsanity.com/paper/2504.13908/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 62 references that are in the map.
- First posted on arXiv 9 Apr 2025.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Zero-shot Text Classification With Generative Language Models](https://mlsanity.com/paper/1912.10165/) (2019)
- [AI Conversational Interviewing: Transforming Surveys with LLMs as Adaptive Interviewers](https://mlsanity.com/paper/2410.01824/) (2024)
- [SmartProbe: A Virtual Moderator for Market Research Surveys](https://mlsanity.com/paper/2305.08271/) (2023)

## Built on by (most cited first, 2 of 2)

- [Can Large Language Models Revolutionize Survey Research? Experiments with Disaster Preparedness Responses](https://mlsanity.com/paper/2605.19229/) (2026)
- [When Surveys Become Conversations: Adaptive Matrix Validation for AI-Assisted Interviews](https://mlsanity.com/paper/2606.24244/) (2026)

## Abstract (arXiv)

Standardized surveys scale efficiently but sacrifice depth, while conversational interviews improve response quality at the cost of scalability and consistency. This study bridges the gap between these methods by introducing a framework for AI-assisted conversational interviewing. To evaluate this framework, we conducted a web survey experiment where 1,800 participants were randomly assigned to AI 'chatbots' which use large language models (LLMs) to dynamically probe respondents for elaboration and interactively code open-ended responses to fixed questions developed by human researchers. We assessed the AI chatbot's performance in terms of coding accuracy, response quality, and respondent experience. Our findings reveal that AI chatbots perform moderately well in live coding even without survey-specific fine-tuning, despite slightly inflated false positive errors due to respondent acquiescence bias. Open-ended responses were more detailed and informative, but this came at a slight cost to respondent experience. Our findings highlight the feasibility of using AI methods such as chatbots enhanced by LLMs to enhance open-ended data collection in web surveys.