Information Discernment in Large Language Models
Not on the map. Why?
Abstract (arXiv)
LLMs are increasingly used with external knowledge sources like the internet. Do they weigh information appropriately -- updating more for reliable sources (source discernment) and more when claims bring priors closer to the truth (truth discernment)? We formalize this as information discernment and introduce Learn2Discern (L2D), an experimental framework and benchmark grounded in three normative axioms with interpretable metrics. To establish external validity, a pre-registered, quota-matched user study (n=299) confirms that real LLM users endorse all three axioms and report that violations reduce their trust and usage intent. Across 13 models and nearly 670K trials, we find consistent failures across both dimensions: models perform near chance on source and truth discernment, rely on source popularity twice as much as source reliability, and update roughly equally whether a claim improves or worsens their position relative to the ground truth. Models integrate external knowledge most effectively on datasets where their priors are already the most accurate. Newer and larger models improve truth discernment but not source discernment, a blind spot that model complexity does not address. We identify simple inference-time interventions that improve both forms of discernment. We release our dataset and survey as a testbed for a core alignment property that scales in importance as LLMs replace traditional search.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 35 references that are in the map. First posted on arXiv 22 May 2026.
Builds on
24 of 35 references.
References · 24
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
- 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
- 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
- 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 452
- 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 348
- 2024Knowledge Conflicts for LLMs: A Surveycited by 283
- 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 264
- 2021Internet-Augmented Dialogue Generationcited by 217
- 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 128
- 2024ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidencecited by 113
- 2024What Large Language Models Know and What People Think They Knowcited by 110
- 2023Combating Misinformation in the Age of LLMs: Opportunities and Challengescited by 95
12 more
- 2024FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"cited by 86
- 2023Context-faithful Prompting for Large Language Modelscited by 80
- 2017The spread of low-credibility content by social botscited by 73
- 2024Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Modelscited by 58
- 2024WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipediacited by 44
- 2024MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generationcited by 44
- 2021Attacking Open-domain Question Answering by Injecting Misinformationcited by 41
- 2024Studying Large Language Model Behaviors Under Context-Memory Conflicts With Real Documentscited by 27
- 2023Accuracy and Political Bias of News Source Credibility Ratings by Large Language Modelscited by 25
- 2024Who's Who: Large Language Models Meet Knowledge Conflicts in Practicecited by 21
- 2025Deep Value Benchmark: Measuring Whether Models Generalize Deep Values or Shallow Preferencescited by 6
- 2024Evaluation of Reliability Criteria for News Publishers with Large Language Modelscited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.