mlsanityData to 7 Oct 2026

The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering

arXiv 2305.16519 ↗First posted 25 May 2023cs.CLcited by 26

Not on the map. Why?

Abstract (arXiv)

Large language models are known to produce output which sounds fluent and convincing, but is also often wrong, e.g. "unfaithful" with respect to a rationale as retrieved from a knowledge base. In this paper, we show that task-based systems which exhibit certain advanced linguistic dialog behaviors, such as lexical alignment (repeating what the user said), are in fact preferred and trusted more, whereas other phenomena, such as pronouns and ellipsis are dis-preferred. We use open-domain question answering systems as our test-bed for task based dialog generation and compare several open- and closed-book models. Our results highlight the danger of systems that appear to be trustworthy by parroting user input while providing an unfaithful response.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 16 of its 29 references that are in the map. First posted on arXiv 25 May 2023.

Builds on

16 of 29 references.

References · 16

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  6. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  7. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
  8. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  9. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  10. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  11. 2022On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?cited by 137
  12. 2021Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answeringcited by 131
4 more
  1. 2022FaithDial: A Faithful Benchmark for Information-Seeking Dialoguecited by 108
  2. 2021TopiOCQA: Open-domain Conversational Question Answering with Topic Switchingcited by 95
  3. 2021Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Featurescited by 74
  4. 2022Trust in Language Grounding: a new AI challenge for human-robot teamscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 8; 2025: 7; 2026: 4.

26 papers.

Sort:

By area:

  1. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 251
  2. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 150
  3. 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systemscited by 115
  4. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  5. 2023Retrieval-augmented Generation to Improve Math Question-Answering: Trade-offs Between Groundedness and Human Preferencecited by 37
  6. 2024CACTUS: Chemistry Agent Connecting Tool-Usage to Sciencecited by 32
  7. 2023From Text to Source: Results in Detecting Large Language Model-Generated Contentcited by 25
  8. 2024From Lived Experience to Insight: Unpacking the Psychological Risks of Using AI Conversational Agentscited by 25
  9. 2023A Diachronic Perspective on User Trust in AI under Uncertaintycited by 23
  10. 2023Assessing Large Language Models on Climate Informationcited by 22
  11. 2025HumT DumT: Measuring and controlling human-like language in LLMscited by 11
  12. 2025Longitudinal Study on Social and Emotional Use of AI Conversational Agentcited by 10
14 more
  1. 2025From tools to thieves: Measuring and understanding public perceptions of AI through crowdsourced metaphorscited by 9
  2. 2025The Next Phase of Scientific Fact-Checking: Advanced Evidence Retrieval from Complex Structured Academic Paperscited by 6
  3. 2023Establishing Trustworthiness: Rethinking Tasks and Model Evaluationcited by 4
  4. 2024SynDy: Synthetic Dynamic Dataset Generation Framework for Misinformation Taskscited by 2
  5. 2024Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systemscited by 2
  6. 2025Adaptive Multi-Agent Response Refinement in Conversational Systemscited by 2
  7. 2024Investigating Content Planning for Navigating Trade-offs in Knowledge-Grounded Dialoguecited by 1
  8. 2025Comparing RAG and GraphRAG for Page-Level Retrieval Question Answering on a Math Textbookcited by 1
  9. 2026Why do we Trust Chatbots? From Normative Principles to Behavioral Driverscited by 1
  10. 2024Making a Long Story Short in Conversation Modelingcited by 0
  11. 2025The illusion of a perfect metric: Why evaluating AI's words is harder than it lookscited by 0
  12. 2026On Improving Faithfulness of Podcasts from Documentscited by 0
  13. 2026Epistemic Trustworthiness in Generative AI: A Normative Framework for Warranted Reliance in High-Stakes Workflowscited by 0
  14. 2026When Readability and Source Retention Diverge: An Evaluability Gap in AI Translationcited by 0