# "What's Up, Doc?": Analyzing How Users Seek Health Information in Large-Scale Conversational AI Datasets

arXiv: https://arxiv.org/abs/2506.21532 · HTML: https://mlsanity.com/paper/2506.21532/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 26 Jun 2025.

## Builds on (most cited first, 14 of 14)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge](https://mlsanity.com/paper/2303.14070/) (2023)
- [OpenChat: Advancing Open-source Language Models with Mixed-Quality Data](https://mlsanity.com/paper/2309.11235/) (2023)
- [Large Language Models are Few-Shot Clinical Information Extractors](https://mlsanity.com/paper/2205.12689/) (2022)
- [MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning](https://mlsanity.com/paper/2406.00922/) (2024)
- [When Large Language Models contradict humans? Large Language Models' Sycophantic Behaviour](https://mlsanity.com/paper/2311.09410/) (2023)
- [The opportunities and risks of large language models in mental health](https://mlsanity.com/paper/2403.14814/) (2024)
- [The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems](https://mlsanity.com/paper/2105.03354/) (2021)
- [Navigating Rifts in Human-LLM Grounding: Study and Benchmark](https://mlsanity.com/paper/2503.13975/) (2025)
- [Can We Trust LLMs? Mitigate Overconfidence Bias in LLMs through Knowledge Transfer](https://mlsanity.com/paper/2405.16856/) (2024)
- [Substance over Style: Evaluating Proactive Conversational Coaching Agents](https://mlsanity.com/paper/2503.19328/) (2025)

## Built on by (most cited first, 5 of 5)

- [MatrAIx: Simulating the World with 8.3 Billion Persona Agents](https://mlsanity.com/paper/2608.04205/) (2026)
- [This Treatment Works, Right? Evaluating LLM Sensitivity to Patient Question Framing in Medical QA](https://mlsanity.com/paper/2604.05051/) (2026)
- [Synthetic Users, Real Differences: an Evaluation Framework for User Simulation in Multi-Turn Conversations](https://mlsanity.com/paper/2605.02624/) (2026)
- [Same Evidence, Different Answer: Auditing Order Sensitivity in Multimodal Large Language Models](https://mlsanity.com/paper/2606.26079/) (2026)
- [TAF-MED: Multi-Turn Safety Refusal Collapse in LLMs Under Declared Self-Treatment Intent](https://mlsanity.com/paper/2608.10258/) (2026)

## Abstract (arXiv)

People are increasingly seeking healthcare information from large language models (LLMs) via interactive chatbots, yet the nature and inherent risks of these conversations remain largely unexplored. In this paper, we filter large-scale conversational AI datasets to achieve HealthChat-11K, a curated dataset of 11K real-world conversations composed of 25K user messages. We use HealthChat-11K and a clinician-driven taxonomy for how users interact with LLMs when seeking healthcare information in order to systematically study user interactions across 21 distinct health specialties. Our analysis reveals insights into the nature of how and why users seek health information, such as common interactions, instances of incomplete context, affective behaviors, and interactions (e.g., leading questions) that can induce sycophancy, underscoring the need for improvements in the healthcare support capabilities of LLMs deployed as conversational AI. Code and artifacts to retrieve our analyses and combine them into a curated dataset can be found here: https://github.com/yahskapar/HealthChat