mlsanityData to 7 Oct 2026

Evaluating Large Language Models for Health-related Queries with Presuppositions

arXiv 2312.08800 ↗First posted 14 Dec 2023cs.CL cs.AI cs.HCcited by 12

Not on the map. Why?

Abstract (arXiv)

As corporations rush to integrate large language models (LLMs) to their search offerings, it is critical that they provide factually accurate information that is robust to any presuppositions that a user may express. In this work, we introduce UPHILL, a dataset consisting of health-related queries with varying degrees of presuppositions. Using UPHILL, we evaluate the factual accuracy and consistency of InstructGPT, ChatGPT, and BingChat models. We find that while model responses rarely disagree with true health claims (posed as questions), they often fail to challenge false claims: responses from InstructGPT agree with 32% of the false claims, ChatGPT 26% and BingChat 23%. As we increase the extent of presupposition in input queries, the responses from InstructGPT and ChatGPT agree with the claim considerably more often, regardless of its veracity. Responses from BingChat, which rely on retrieved webpages, are not as susceptible. Given the moderate factual accuracy, and the inability of models to consistently correct false assumptions, our work calls for a careful assessment of current LLMs for use in high-stakes scenarios.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 44 of its 73 references that are in the map. First posted on arXiv 14 Dec 2023.

Builds on

44 of 73 references.

References · 44

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  8. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  9. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  10. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  11. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  12. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
32 more
  1. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  2. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  3. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  4. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  5. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 402
  6. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  7. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  8. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  9. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  10. 2020FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarizationcited by 292
  11. 2022Teaching language models to support answers with verified quotescited by 290
  12. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  13. 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 284
  14. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 247
  15. 2021QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarizationcited by 233
  16. 2020Explainable Automated Fact-Checking for Public Health Claimscited by 209
  17. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
  18. 2022Can large language models reason about medical questions?cited by 194
  19. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 150
  20. 2020Evaluating Factuality in Generation with Dependency-level Entailmentcited by 122
  21. 2021Entity-level Factual Consistency of Abstractive Text Summarizationcited by 118
  22. 2019Assessing The Factual Accuracy of Generated Textcited by 103
  23. 2023Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debatecited by 92
  24. 2023Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queriescited by 82
  25. 2023ChatGPT as a Factual Inconsistency Evaluator for Text Summarizationcited by 75
  26. 2022CREPE: Open-Domain Question Answering with False Presuppositionscited by 70
  27. 2020RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databasescited by 63
  28. 2022(QA)2: Question Answering with Questionable Assumptionscited by 51
  29. 2020Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraintscited by 48
  30. 2021Attacking Open-domain Question Answering by Injecting Misinformationcited by 41
  31. 2020Adversarial NLI for Factual Correctness in Text Summarisation Modelscited by 13
  32. 2022Monant Medical Misinformation Dataset: Mapping Articles to Fact-Checked Claimscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 5; 2026: 3.

12 papers.

  1. 2024MedHalu: Hallucinations in Responses to Healthcare Queries by Large Language Modelscited by 20
  2. 2024BiMediX2: Bio-Medical EXpert LMM for Diverse Medical Modalitiescited by 20
  3. 2024Enhancing Recommendation Diversity by Re-ranking with Large Language Modelscited by 19
  4. 2024Search Engines in an AI Era: The False Promise of Factual and Verifiable Source-Cited Responsescited by 15
  5. 2025DeepTRACE: Auditing Deep Research AI Systems for Tracking Reliability Across Citations and Evidencecited by 10
  6. 2025MythTriage: Scalable Detection of Opioid Use Disorder Myths on a Video-Sharing Platformcited by 3
  7. 2025Who's Asking? Simulating Role-Based Questions for Conversational AI Evaluationcited by 1
  8. 2025Zoom In Disparities in Healthcare LLM Q&Acited by 1
  9. 2026XYBench: Can LLMs Respond Pragmatically to Queries with Misconceptions?cited by 1
  10. 2025Assessing Web Search Credibility and Response Groundedness in Chat Assistantscited by 0
  11. 2026MedRedFlag: Investigating how LLMs Redirect Misconceptions in Real-World Health Communicationcited by 0
  12. 2026Towards Emotion Consistency Analysis of Large Language Models in Emotional Conversational Contextscited by 0