# Designing Effective Interview Chatbots: Automatic Chatbot Profiling and Design Suggestion Generation for Chatbot Debugging

arXiv: https://arxiv.org/abs/2104.04842 · HTML: https://mlsanity.com/paper/2104.04842/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2021.

## Built on by (most cited first, 12 of 12)

- [Approximating Online Human Evaluation of Social Chatbots with Prompting](https://mlsanity.com/paper/2304.05253/) (2023)
- [What is Stigma Attributed to? A Theory-Grounded, Expert-Annotated Interview Corpus for Demystifying Mental-Health Stigma](https://mlsanity.com/paper/2505.12727/) (2025)
- [Requirements Elicitation Follow-Up Question Generation](https://mlsanity.com/paper/2507.02858/) (2025)
- [On LLM Wizards: Identifying Large Language Models' Behaviors for Wizard of Oz Experiments](https://mlsanity.com/paper/2407.08067/) (2024)
- [Deconstructing Depression Stigma: Integrating AI-driven Data Collection and Analysis with Causal Knowledge Graphs](https://mlsanity.com/paper/2502.06075/) (2025)
- [AI-Enabled Conversational Journaling for Advancing Parkinson's Disease Symptom Tracking](https://mlsanity.com/paper/2503.03532/) (2025)
- [Exploring the Human-LLM Synergy in Advancing Theory-driven Qualitative Analysis](https://mlsanity.com/paper/2405.05758/) (2024)
- [HumAIne-Chatbot: Real-Time Personalized Conversational AI via Reinforcement Learning](https://mlsanity.com/paper/2509.04303/) (2025)
- [Plausibility as Failure: How LLMs and Humans Co-Construct Epistemic Error](https://mlsanity.com/paper/2512.16750/) (2025)
- [Modular AI-Powered Interviewer with Dynamic Question Generation and Expertise Profiling](https://mlsanity.com/paper/2601.11534/) (2025)
- [Polite But Boring? Trade-offs Between Engagement and Psychological Reactance to Chatbot Feedback Styles](https://mlsanity.com/paper/2601.20683/) (2026)
- [From Chat to Interview: Agentic Requirements Elicitation with an Experience Ontology](https://mlsanity.com/paper/2605.05828/) (2026)

## Abstract (arXiv)

Recent studies show the effectiveness of interview chatbots for information elicitation. However, designing an effective interview chatbot is non-trivial. Few tools exist to help designers design, evaluate, and improve an interview chatbot iteratively. Based on a formative study and literature reviews, we propose a computational framework for quantifying the performance of interview chatbots. Incorporating the framework, we have developed iChatProfile, an assistive chatbot design tool that can automatically generate a profile of an interview chatbot with quantified performance metrics and offer design suggestions for improving the chatbot based on such metrics. To validate the effectiveness of iChatProfile, we designed and conducted a between-subject study that compared the performance of 10 interview chatbots designed with or without using iChatProfile. Based on the live chats between the 10 chatbots and 1349 users, our results show that iChatProfile helped the designers build significantly more effective interview chatbots, improving both interview quality and user experience.