# To Know or Not To Know? Analyzing Self-Consistency of Large Language Models under Ambiguity

arXiv: https://arxiv.org/abs/2407.17125 · HTML: https://mlsanity.com/paper/2407.17125/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 9 of its 22 references that are in the map.
- First posted on arXiv 24 Jul 2024.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 9 of 9)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [TrustLLM: Trustworthiness in Large Language Models](https://mlsanity.com/paper/2401.05561/) (2024)
- [Aligning Language Models to Explicitly Handle Ambiguity](https://mlsanity.com/paper/2404.11972/) (2024)
- [Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Method](https://mlsanity.com/paper/2310.17918/) (2023)

## Built on by (most cited first, 8 of 8)

- [Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps](https://mlsanity.com/paper/2502.14829/) (2025)
- [Verify-in-the-Graph: Entity Disambiguation Enhancement for Complex Claim Verification with Interactive Graph Representation](https://mlsanity.com/paper/2505.22993/) (2025)
- [Are Economists Always More Introverted? Analyzing Consistency in Persona-Assigned LLMs](https://mlsanity.com/paper/2506.02659/) (2025)
- [Clarify, Abstain or Answer? Strategising in Conversation with Belief-Augmented Generation](https://mlsanity.com/paper/2605.25831/) (2026)
- [LLM-Based Config Synthesis requires Disambiguation](https://mlsanity.com/paper/2507.12443/) (2025)
- [Math Natural Language Inference: this should be easy!](https://mlsanity.com/paper/2507.23063/) (2025)
- [Can LLMs Detect Ambiguous Plural Reference? An Analysis of Split-Antecedent and Mereological Reference](https://mlsanity.com/paper/2510.04581/) (2025)
- [Semantic Abstraction for Natural Language Inference: a Methodological Framework for Discovering and Compensating Semantic Knowledge and Reasoning Gaps in Large Language Models](https://mlsanity.com/paper/2609.26610/) (2026)

## Abstract (arXiv)

One of the major aspects contributing to the striking performance of large language models (LLMs) is the vast amount of factual knowledge accumulated during pre-training. Yet, many LLMs suffer from self-inconsistency, which raises doubts about their trustworthiness and reliability. This paper focuses on entity type ambiguity, analyzing the proficiency and consistency of state-of-the-art LLMs in applying factual knowledge when prompted with ambiguous entities. To do so, we propose an evaluation protocol that disentangles knowing from applying knowledge, and test state-of-the-art LLMs on 49 ambiguous entities. Our experiments reveal that LLMs struggle with choosing the correct entity reading, achieving an average accuracy of only 85%, and as low as 75% with underspecified prompts. The results also reveal systematic discrepancies in LLM behavior, showing that while the models may possess knowledge, they struggle to apply it consistently, exhibit biases toward preferred readings, and display self-inconsistencies. This highlights the need to address entity ambiguity in the future for more trustworthy LLMs.