mlsanityData to 13 Oct 2026

Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data

arXiv 2301.05843 ↗First posted 14 Jan 2023cs.HC cs.AI cs.CLcited by 48

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) provide a new way to build chatbots by accepting natural language prompts. Yet, it is unclear how to design prompts to power chatbots to carry on naturalistic conversations while pursuing a given goal, such as collecting self-report data from users. We explore what design factors of prompts can help steer chatbots to talk naturally and collect data reliably. To this aim, we formulated four prompt designs with different structures and personas. Through an online study (N = 48) where participants conversed with chatbots driven by different designs of prompts, we assessed how prompt designs and conversation topics affected the conversation flows and users' perceptions of chatbots. Our chatbots covered 79% of the desired information slots during conversations, and the designs of prompts and topics significantly influenced the conversation flows and the data collection performance. We discuss the opportunities and challenges of building chatbots with LLMs.

Key facts. Cited by 48 papers in the map (data through 13 Oct 2026). Builds on 34 of its 124 references that are in the map. First posted on arXiv 14 Jan 2023.

Builds on

34 of 124 references.

References · 34

  1. 2020Language Models are Few-Shot Learnerscited by 32,320
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,317
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,348
  5. 2021Program Synthesis with Large Language Modelscited by 3,573
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,500
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,229
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,424
  9. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,249
  10. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,191
  11. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
  12. 2019Neural Text Generation with Unlikelihood Trainingcited by 494
22 more
  1. 2020Towards a Human-like Open-Domain Chatbotcited by 469
  2. 2022Social Simulacra: Creating Populated Prototypes for Social Computing Systemscited by 301
  3. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 266
  4. 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 245
  5. 2021Design Guidelines for Prompt Engineering Text-to-Image Generative Modelscited by 218
  6. 2018The Design and Implementation of XiaoIce, an Empathetic Social Chatbotcited by 172
  7. 2019What makes a good conversation? How controllable attributes affect human judgmentscited by 140
  8. 2021Dialogue State Tracking with a Language Model using Schema-Driven Promptingcited by 94
  9. 2022The AI Teacher Test: Measuring the Pedagogical Ability of Blender and GPT-3 in Educational Dialoguescited by 78
  10. 2021What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformerscited by 76
  11. 2022Enabling Conversational Interaction with Mobile UI using Large Language Modelscited by 66
  12. 2021Zero-Shot Dialogue State Tracking via Cross-Task Transfercited by 52
  13. 2020A Taxonomy of Empathetic Response Intents in Human Social Conversationscited by 51
  14. 2020Effects of Persuasive Dialogues: Testing Bot Identities and Inquiry Strategiescited by 29
  15. 2022Building a Role Specified Open-Domain Dialogue System Leveraging Large-Scale Language Modelscited by 26
  16. 2020If I Hear You Correctly: Building and Evaluating Interview Chatbots with Active Listening Skillscited by 20
  17. 2022LAD: Language Models as Data for Zero-Shot Dialogcited by 18
  18. 2019Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questionscited by 16
  19. 2021An Evaluation of Generative Pre-Training Model-based Therapy Chatbot for Caregiverscited by 15
  20. 2020Developing a Personality Model for Speech-based Conversational Agents Using the Psycholexical Approachcited by 8
  21. 2020Submitting surveys via a conversational interface: an evaluation of user acceptance and approach effectivenesscited by 6
  22. 2022MyMove: Facilitating Older Adults to Collect In-Situ Activity Labels on a Smartwatch with Speechcited by 5

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 11; 2024: 24; 2025: 10; 2026: 3.

48 papers.

Sort:

By area:

  1. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 92
  2. 2024Safeguarding Large Language Models: A Surveycited by 76
  3. 2023EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteriacited by 74
  4. 2024Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCachecited by 71
  5. 2024Training-Free Long-Context Scaling of Large Language Modelscited by 62
  6. 2023MindfulDiary: Harnessing Large Language Model to Support Psychiatric Patients' Journalingcited by 50
  7. 2024Prompt Stealing Attacks Against Large Language Modelscited by 40
  8. 2023LLM-empowered Chatbots for Psychiatrist and Patient Simulation: Application and Evaluationcited by 33
  9. 2023ChaCha: Leveraging Large Language Models to Prompt Children to Share Their Emotions about Personal Eventscited by 31
  10. 2023MindShift: Leveraging Large Language Models for Mental-States-Based Problematic Smartphone Use Interventioncited by 16
  11. 2024Understanding the Impact of Long-Term Memory on Self-Disclosure with Large Language Model-Driven Chatbots for Public Health Interventioncited by 15
  12. 2024Position: Towards Bidirectional Human-AI Alignmentcited by 15
36 more
  1. 2024Large Language Model for Mental Health: A Systematic Reviewcited by 14
  2. 2024The Impact of Prompt Programming on Function-Level Code Generationcited by 13
  3. 2024CEBench: A Benchmarking Toolkit for the Cost-Effectiveness of LLM Pipelinescited by 11
  4. 2024Evaluation of an LLM in Identifying Logical Fallacies: A Call for Rigor When Adopting LLMs in HCI Researchcited by 10
  5. 2024An Empirical Categorization of Prompting Techniques for Large Language Models: A Practitioner's Guidecited by 7
  6. 2023PlanFitting: Personalized Exercise Planning with Large Language Model-driven Conversational Agentcited by 6
  7. 2024Ain't Misbehavin' -- Using LLMs to Generate Expressive Robot Behavior in Conversations with the Tabletop Robot Harucited by 5
  8. 2024ExploreSelf: Fostering User-driven Exploration and Reflection on Personal Challenges with Adaptive Guidance by Large Language Modelscited by 5
  9. 2024Advancing Conversational Psychotherapy: Integrating Privacy, Dual-Memory, and Domain Expertise with Large Language Modelscited by 5
  10. 2025Telephone Surveys Meet Conversational AI: Evaluating a LLM-Based Telephone Survey System at Scalecited by 5
  11. 2024Exploring Conversational Agents as an Effective Tool for Measuring Cognitive Biases in Decision-Makingcited by 4
  12. 2024Unimib Assistant: designing a student-friendly RAG-based chatbot for all their needscited by 4
  13. 2025AI-Enabled Conversational Journaling for Advancing Parkinson's Disease Symptom Trackingcited by 4
  14. 2025DiaCBT: A Long-Periodic Dialogue Corpus Guided by Cognitive Conceptualization Diagram for CBT-based Psychological Counselingcited by 4
  15. 2025ClearFairy: Capturing Creative Workflows through Decision Structuring, In-Situ Questioning, and Rationale Inferencecited by 4
  16. 2024Fine-tuning with HED-IT: The impact of human post-editing for dialogical language modelscited by 3
  17. 2024A Systematic Review of Generative AI for Teaching and Learning Practicecited by 3
  18. 2025Pet-Bench: Benchmarking the Abilities of Large Language Models as E-Pets in Social Network Servicescited by 3
  19. 2026AI and My Values: User Perceptions of LLMs' Ability to Extract, Embody, and Explain Human Values from Casual Conversationscited by 2
  20. 2023Talk2Care: Facilitating Asynchronous Patient-Provider Communication with Large-Language-Modelcited by 1
  21. 2023Comparing Large Language Model AI and Human-Generated Coaching Messages for Behavioral Weight Losscited by 1
  22. 2024A General-purpose AI Avatar in Healthcarecited by 1
  23. 2024SeSaMe: A Framework to Simulate Self-Reported Ground Truth for Mental Health Sensing Studiescited by 1
  24. 2024The Evolution of Learning: Assessing the Transformative Impact of Generative AI on Higher Educationcited by 1
  25. 2024Design Proteins Using Large Language Models: Enhancements and Comparative Analysescited by 1
  26. 2023Keeping Users Engaged During Repeated Administration of the Same Questionnaire: Using Large Language Models to Reliably Diversify Questionscited by 0
  27. 2023Saturn Platform: Foundation Model Operations and Generative AI for Financial Servicescited by 0
  28. 2024Context Matters: Pushing the Boundaries of Open-Ended Answer Generation with Graph-Structured Knowledge Contextcited by 0
  29. 2024Exploring a Physics-Informed Decision Transformer for Distribution System Restoration: Methodology and Performance Analysiscited by 0
  30. 2025"It Felt Like I Was Left in the Dark": Exploring Information Needs and Design Opportunities for Family Caregivers of Older Adult Patients in Critical Care Settingscited by 0
  31. 2025Mic Drop or Data Flop? Evaluating the Fitness for Purpose of AI Voice Interviewers for Data Collection within Quantitative & Qualitative Research Contextscited by 0
  32. 2025Investigating Student Interaction Patterns with Large Language Model-Powered Course Assistants in Computer Science Coursescited by 0
  33. 2025Autiverse: Eliciting Autistic Adolescents' Daily Narratives through AI-guided Multimodal Journalingcited by 0
  34. 2025Evaluating the Challenges of LLMs in Real-world Medical Follow-up: A Comparative Study and An Optimized Frameworkcited by 0
  35. 2026Communication Policy Evolution for Proactive LLM Agentscited by 0
  36. 2026From Momentary Emotion Inference to Sustained Emotion Support: Evaluating a Companion Agent in a Longitudinal Studycited by 0