mlsanityData to 13 Oct 2026

Conversation as Measurement in Clinical Encounters: Observable Phase Structure, Partially Observable Patient State

arXiv 2608.08868 ↗First posted 9 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Many modern AI systems analyze conversational traces to infer aspects of human interaction and state, implicitly assuming that such information is recoverable from conversation. We study observability: whether a target is recoverable from conversational transcripts alone. Observability is difficult to assess because transcripts may provide only a partial view of many targets, and large-scale analysis requires model-based annotation, making true limits of the conversational signal hard to distinguish from annotator error. We therefore study clinical encounters, where patient-reported outcome measures (PROMs) provide an external anchor for patient state, and visits follow broadly structured patterns. We study observability of patient state and conversational phase structure using 439 real-world clinical encounter transcripts spanning 134 hours, including 245 ENT transcripts paired with 273 PROM surveys. We operationalize patient state using PROM scores for voice, cough, and swallowing; phase structure using conversational phase segmentation. To make these analyses credible at scale, we use a PHI-compliant GPT-5 deployment for transcript annotation and conduct 40 hours of manual validation, reducing the risk that apparent limits of observability simply reflect annotator error. Our core finding is an observability asymmetry: phase structure is observable and useful for characterizing clinical encounter organization, while patient state is only partially observable, even in a setting designed to elicit patient symptoms and experiences, cautioning against transcript-only inference of human state.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 9 of its 42 references that are in the map. First posted on arXiv 9 Aug 2026.

Builds on

9 of 42 references.

References · 9

  1. 2022Training language models to follow instructions with human feedbackcited by 15,856
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,492
  3. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 857
  4. 2024MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialoguescited by 250
  5. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 198
  6. 2022In-Context Learning for Few-Shot Dialogue State Trackingcited by 47
  7. 2023ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?cited by 38
  8. 2023BotChat: Evaluating LLMs' Capabilities of Having Multi-Turn Dialoguescited by 34
  9. 2023DialogBench: Evaluating LLMs as Human-like Dialogue Systemscited by 31

Built on by

Not cited yet.