# Telephone Surveys Meet Conversational AI: Evaluating a LLM-Based Telephone Survey System at Scale

arXiv: https://arxiv.org/abs/2502.20140 · HTML: https://mlsanity.com/paper/2502.20140/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 43 references that are in the map.
- First posted on arXiv 27 Feb 2025.

## Builds on (most cited first, 13 of 13)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models](https://mlsanity.com/paper/2308.01263/) (2023)
- [Moral Foundations of Large Language Models](https://mlsanity.com/paper/2310.15337/) (2023)
- [When Large Language Models contradict humans? Large Language Models' Sycophantic Behaviour](https://mlsanity.com/paper/2311.09410/) (2023)
- [Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data](https://mlsanity.com/paper/2301.05843/) (2023)
- [A Full-duplex Speech Dialogue Scheme Based On Large Language Models](https://mlsanity.com/paper/2405.19487/) (2024)
- [Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue Systems](https://mlsanity.com/paper/2205.15060/) (2022)
- [What should I Ask: A Knowledge-driven Approach for Follow-up Questions Generation in Conversational Surveys](https://mlsanity.com/paper/2205.10977/) (2022)
- [AI Conversational Interviewing: Transforming Surveys with LLMs as Adaptive Interviewers](https://mlsanity.com/paper/2410.01824/) (2024)
- [LLM-Enhanced Dialogue Management for Full-Duplex Spoken Dialogue Systems](https://mlsanity.com/paper/2502.14145/) (2025)
- [Adopting AI: How Familiarity Breeds Both Trust and Contempt](https://mlsanity.com/paper/2305.01405/) (2023)
- [Unpacking Human-AI Interaction in Safety-Critical Industries: A Systematic Literature Review](https://mlsanity.com/paper/2310.03392/) (2023)
- [Collecting Qualitative Data at Scale with Large Language Models: A Case Study](https://mlsanity.com/paper/2309.10187/) (2023)

## Built on by (most cited first, 5 of 5)

- [Are LLMs Empathetic to All? Investigating the Influence of Multi-Demographic Personas on a Model's Empathy](https://mlsanity.com/paper/2510.10328/) (2025)
- [VoiceAgentEval: A Dual-Dimensional Benchmark for Expert-Level Intelligent Voice-Agent Evaluation of Xbench's Professional-Aligned Series](https://mlsanity.com/paper/2510.21244/) (2025)
- [Mic Drop or Data Flop? Evaluating the Fitness for Purpose of AI Voice Interviewers for Data Collection within Quantitative & Qualitative Research Contexts](https://mlsanity.com/paper/2509.01814/) (2025)
- [Generating Diverse Personas for User Simulators to Test Interview Dialogue Systems](https://mlsanity.com/paper/2608.19549/) (2026)
- [InterView-C: A Synchronized Multimodal Corpus of VR Avatar-Mediated Survey Interviews](https://mlsanity.com/paper/2610.10145/) (2026)

## Abstract (arXiv)

Telephone surveys remain a valuable tool for gathering insights but typically require substantial resources in training and coordinating human interviewers. This work presents an AI-driven telephone survey system integrating text-to-speech (TTS), a large language model (LLM), and speech-to-text (STT) that mimics the versatility of human-led interviews (full-duplex dialogues) at scale. We tested the system across two populations, a pilot study in the United States (n = 75) and a large-scale deployment in Peru (n = 2,739), inviting participants via web-based links and contacting them via direct phone calls. The AI agent successfully administered open-ended and closed-ended questions, handled basic clarifications, and dynamically navigated branching logic, allowing fast large-scale survey deployment without interviewer recruitment or training. Our findings demonstrate that while the AI system's probing for qualitative depth was more limited than human interviewers, overall data quality approached human-led standards for structured items. This study represents one of the first successful large-scale deployments of an LLM-based telephone interviewer in a real-world survey context. The AI-powered telephone survey system has the potential for expanding scalable, consistent data collecting across market research, social science, and public opinion studies, thus improving operational efficiency while maintaining appropriate data quality for research.