# What Makes a Good Response? An Empirical Analysis of Quality in Qualitative Interviews

arXiv: https://arxiv.org/abs/2604.05163 · HTML: https://mlsanity.com/paper/2604.05163/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 42 references that are in the map.
- First posted on arXiv 6 Apr 2026.

## Builds on (most cited first, 7 of 7)

- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Frequency Explains the Inverse Correlation of Large Language Models' Size, Training Data Amount, and Surprisal's Fit to Reading Times](https://mlsanity.com/paper/2402.02255/) (2024)
- [Bayesian Preference Elicitation with Language Models](https://mlsanity.com/paper/2403.05534/) (2024)
- [BED-LLM: Intelligent Information Gathering with LLMs and Bayesian Experimental Design](https://mlsanity.com/paper/2508.21184/) (2025)
- [Chatbots for Data Collection in Surveys: A Comparison of Four Theory-Based Interview Probes](https://mlsanity.com/paper/2503.08582/) (2025)
- [SparkMe: Adaptive Semi-Structured Interviewing for Qualitative Insight Discovery](https://mlsanity.com/paper/2602.21136/) (2026)
- [PCQPR: Proactive Conversational Question Planning with Reflection](https://mlsanity.com/paper/2410.01363/) (2024)

## Built on by (most cited first, 1 of 1)

- [InterviewPlayground: A Validated Simulation Environment for Evaluating AI Interviewers](https://mlsanity.com/paper/2610.12023/) (2026)

## Abstract (arXiv)

Qualitative interviews provide essential insights into human experiences when they elicit high-quality responses. While qualitative and NLP researchers have proposed various measures of interview quality, these measures lack validation that high-scoring responses actually contribute to the study's goals. In this work, we identify, implement, and evaluate 10 proposed measures of interview response quality to determine which are actually predictive of a response's contribution to the study findings. To conduct our analysis, we introduce the Qualitative Interview Corpus, a newly constructed dataset of 343 interview transcripts with 16,940 participant responses from 14 real research projects. We find that direct relevance to a key research question is the strongest predictor of response quality. We additionally find that two measures commonly used to evaluate NLP interview systems, clarity and surprisal-based informativeness, are not predictive of response quality. Our work provides analytic insights and grounded, scalable metrics to inform the design of qualitative studies and the evaluation of automated interview systems.