mlsanityData to 14 Oct 2026

A knowledge-guided agentic framework for mitigating patient-context ambiguity in health queries

arXiv 2608.19875 ↗First posted 20 Aug 2026cs.CL cs.AIcited by 0

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

Patients often submit short, underspecified queries to healthcare chatbots that lack the patient-specific information needed to determine an appropriate response. Although these queries may be linguistically clear, they can support multiple plausible answers depending on undisclosed factors such as symptoms, diagnoses, medications, allergies, or dietary restrictions. A language model answering such a query directly may therefore rely on unsupported assumptions about the patient. We introduce a knowledge-guided agentic framework for mitigating patient-context ambiguity before final response generation. The framework operates between the patient and an otherwise unchanged downstream language model. It interprets the initial query, uses a task-specific knowledge graph to construct a set of plausible hypotheses, identifies the missing patient-context variables needed to distinguish among them, and asks targeted follow-up questions. The original query and the acquired context are then combined into a clarified prompt for the downstream model. We evaluated the framework across five language models using two controlled ambiguity-mitigation benchmarks: diagnosis retrieval from 1,034 symptom queries with clinically relevant evidence systematically masked, and dietary-safety classification from 487 queries with decisive health context omitted. The framework was compared with direct answering of the underspecified query and with rephrasing the same query without acquiring new patient information. In diagnosis retrieval, it increased overall exact Top-1 accuracy by at least 57.1 percentage points and selective exact Recall@5 by at least 77.7 percentage points across the five evaluated models compared with direct prompting. In dietary-safety classification, it improved accuracy across all five models and achieved the highest Matthews correlation coefficient for four...

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 24 of its 35 references that are in the map. First posted on arXiv 20 Aug 2026.

Builds on

24 of 35 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Agentic Medical Knowledge Graphs Enhance Medical Question Answering: Bridging the Gap Between LLMs and Evolving Medical Knowledge18 Feb

Other references · 23

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  3. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  4. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 390
  5. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 293
  6. 2019ScispaCy: Fast and Robust Models for Biomedical Natural Language Processingcited by 244
  7. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 234
  8. 2024MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoningcited by 154
  9. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 126
  10. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  11. 2023Selectively Answering Ambiguous Questionscited by 66
  12. 2024Aligning Language Models to Explicitly Handle Ambiguitycited by 56
11 more
  1. 2024CLAMBER: A Benchmark of Identifying and Clarifying Ambiguous Information Needs in Large Language Modelscited by 54
  2. 2021Joint Passage Ranking for Diverse Multi-Answer Retrievalcited by 43
  3. 2021Launching into clinical space with medspaCy: a new clinical text processing toolkit in Pythoncited by 40
  4. 2022Task Ambiguity in Humans and Language Modelscited by 28
  5. 2023Asking Clarification Questions to Handle Ambiguity in Open-Domain QAcited by 26
  6. 2020Answering Ambiguous Questions through Generative Evidence Fusion and Round-Trip Predictioncited by 19
  7. 2025Large language models provide unsafe answers to patient-posed medical questionscited by 18
  8. 2024STYLE: Improving Domain Transferability of Asking Clarification Questions in Large Language Model Powered Conversational Agentscited by 17
  9. 2024A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluationscited by 16
  10. 2025CLEAR-KGQA: Clarification-Enhanced Ambiguity Resolution for Knowledge Graph Question Answeringcited by 3
  11. 2026Mind the Ambiguity: Aleatoric Uncertainty Quantification in LLMs for Safe Medical Question Answeringcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026MedClarify: An information-seeking AI agent for medical diagnosis with case-specific follow-up questions
  2. 2025HealthContradict: Evaluating Biomedical Knowledge Conflicts in Language Models
  3. 2026MedDialogRubrics: A Comprehensive Benchmark and Evaluation Framework for Multi-turn Medical Consultations in Large Language Models
  4. 2026Evidence, Logic, and Compliance: Multi-Agent Structured Graph Reasoning with Expert Arbitration for Medical Referral
  5. 2024LLM-MedQA: Enhancing Medical Question Answering through Case Studies in Large Language Models
  6. 2026CareGuardAI: Context-Aware Multi-Agent Guardrails for Clinical Safety & Hallucination Mitigation in Patient-Facing LLMs
  7. 2026EHRNavigator: A Multi-Agent System for Patient-Level Clinical Question Answering over Heterogeneous Electronic Health Records
  8. 2025Enhancing the Medical Context-Awareness Ability of LLMs via Multifaceted Self-Refinement Learning