Epistemic Alignment: A Mediating Framework for User-LLM Knowledge Delivery
Not on the map. Why?
Abstract (arXiv)
LLMs increasingly serve as tools for knowledge acquisition, yet users cannot effectively specify how they want information presented. When users request that LLMs "cite reputable sources," "express appropriate uncertainty," or "include multiple perspectives," they discover that current interfaces provide no structured way to articulate these preferences. The result is prompt sharing folklore: community-specific copied prompts passed through trust relationships rather than based on measured efficacy. We propose the Epistemic Alignment Framework, a set of ten challenges in knowledge transmission derived from the philosophical literature of epistemology, concerning issues such as evidence quality assessment and calibration of testimonial reliance. The framework serves as a structured intermediary between user needs and system capabilities, creating a common vocabulary to bridge the gap between what users want and what systems deliver. Through a thematic analysis of custom prompts and personalization strategies shared on online communities where these issues are actively discussed, we find users develop elaborate workarounds to address each of the challenges. We then apply our framework to two prominent model providers, OpenAI and Anthropic, through content analysis of their documented policies and product features. Our analysis shows that while these providers have partially addressed the challenges we identified, they fail to establish adequate mechanisms for specifying epistemic preferences, lack transparency about how preferences are implemented, and offer no verification tools to confirm whether preferences were followed. For AI developers, the Epistemic Alignment Framework offers concrete guidance for supporting diverse approaches to knowledge; for users, it works toward information delivery that aligns with their specific needs rather than defaulting to one-size-fits-all approaches.
Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 34 of its 64 references that are in the map. First posted on arXiv 1 Apr 2025.
Builds on
34 of 64 references.
References · 34
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2022Survey of Hallucination in Natural Language Generationcited by 2,056
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2023Enabling Large Language Models to Generate Text with Citationscited by 584
- 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
- 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
- 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
- 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
- 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
- 2024Knowledge Conflicts for LLMs: A Surveycited by 274
- 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
22 more
- 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
- 2024A Roadmap to Pluralistic Alignmentcited by 175
- 2024Fine-grained Hallucination Detection and Editing for Language Modelscited by 146
- 2024Language Models with Conformal Factuality Guaranteescited by 128
- 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 124
- 2024Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaborationcited by 111
- 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 107
- 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 100
- 2024Personalization of Large Language Models: A Surveycited by 100
- 2024Tug-of-War Between Knowledge: Exploring and Resolving Knowledge Conflicts in Retrieval-Augmented Language Modelscited by 93
- 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 85
- 2024Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Datacited by 65
- 2023The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensivenesscited by 63
- 2023Resolving Knowledge Conflicts in Large Language Modelscited by 49
- 2024Can AI Assistants Know What They Don't Know?cited by 47
- 2024NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Modelscited by 47
- 2024A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Taskscited by 46
- 2023Citation: A Key to Building Responsible and Accountable Large Language Modelscited by 20
- 2024User Intent Recognition and Satisfaction with Large Language Models: A User Study with ChatGPTcited by 19
- 2025Citations and Trust in LLM Generated Responsescited by 17
- 2024DebateQA: Evaluating Question Answering on Debatable Knowledgecited by 12
- 2024Generative AI in Education: A Study of Educators' Awareness, Sentiments, and Influencing Factorscited by 10
Built on by
Citing papers per year: 2025: 4; 2026: 10.
14 papers.
Sort:
By area:
- 2025Beyond One-Way Influence: Bidirectional Opinion Dynamics in Multi-Turn Human-LLM Interactionscited by 5
- 2026Positive Alignment: Artificial Intelligence for Human Flourishingcited by 5
- 2026When Planning Fails Despite Correct Execution: On Epistemic Calibration for LLM-Based Multi-Agent Systemscited by 3
- 2025Collaborative Causal Sensemaking: Closing the Complementarity Gap in Human-AI Decision Supportcited by 2
- 2026Frictive Policy Optimization for LLMs: Epistemic Intervention, Risk-Sensitive Control, and Reflective Alignmentcited by 2
- 2025Fragments to Facts: Partial-Information Fragment Inference from LLMscited by 0
- 2025Matching Game Preferences Through Dialogical Large Language Models: A Perspectivecited by 0
- 2026From Sycophancy to Sensemaking: Premise Governance for Human-AI Decision Makingcited by 0
- 2026Epistemic Observability in Language Modelscited by 0
- 2026The LLM Fallacy: Misattribution in AI-Assisted Cognitive Workflowscited by 0
- 2026Saying More Than They Know: A Framework for Quantifying Epistemic-Rhetorical Miscalibration in Large Language Modelscited by 0
- 2026How Researchers Navigate Accountability, Transparency, and Trust When Using AI Tools in Early-Stage Research: A Think-Aloud Studycited by 0
2 more
- 2026When English Rewrites Local Knowledge: Global Narrative Dominance in Large Language Models28 May 2026
- 2026When Planning Fails Despite Correct Execution: On Epistemic Calibration for LLM-Based Multi-Agent Systems22 May 2026
- 2026Positive Alignment: Artificial Intelligence for Human Flourishing11 May 2026
- 2026Can LLMs Emulate Human Belief Dynamics?5 May 2026
- 2026Frictive Policy Optimization for LLMs: Epistemic Intervention, Risk-Sensitive Control, and Reflective Alignment28 Apr 2026
- 2026How Researchers Navigate Accountability, Transparency, and Trust When Using AI Tools in Early-Stage Research: A Think-Aloud Study25 Apr 2026
- 2026The LLM Fallacy: Misattribution in AI-Assisted Cognitive Workflows16 Apr 2026
- 2026Saying More Than They Know: A Framework for Quantifying Epistemic-Rhetorical Miscalibration in Large Language Models27 Mar 2026
- 2026Epistemic Observability in Language Models20 Mar 2026
- 2026From Sycophancy to Sensemaking: Premise Governance for Human-AI Decision Making2 Feb 2026
- 2025Collaborative Causal Sensemaking: Closing the Complementarity Gap in Human-AI Decision Support8 Dec 2025
- 2025Beyond One-Way Influence: Bidirectional Opinion Dynamics in Multi-Turn Human-LLM Interactions22 Oct 2025
- 2025Beyond One-Way Influence: Bidirectional Opinion Dynamics in Multi-Turn Human-LLM Interactionscited by 5
- 2026Positive Alignment: Artificial Intelligence for Human Flourishingcited by 5
- 2026Frictive Policy Optimization for LLMs: Epistemic Intervention, Risk-Sensitive Control, and Reflective Alignmentcited by 2
- 2025Matching Game Preferences Through Dialogical Large Language Models: A Perspectivecited by 0
- 2026From Sycophancy to Sensemaking: Premise Governance for Human-AI Decision Makingcited by 0
- 2026Epistemic Observability in Language Modelscited by 0
- 2026The LLM Fallacy: Misattribution in AI-Assisted Cognitive Workflowscited by 0
- 2026Saying More Than They Know: A Framework for Quantifying Epistemic-Rhetorical Miscalibration in Large Language Modelscited by 0
- 2026Can LLMs Emulate Human Belief Dynamics?cited by 0
- 2026When English Rewrites Local Knowledge: Global Narrative Dominance in Large Language Modelscited by 0
- 2026When English Rewrites Local Knowledge: Global Narrative Dominance in Large Language Models28 May 2026
- 2026Positive Alignment: Artificial Intelligence for Human Flourishing11 May 2026
- 2026Can LLMs Emulate Human Belief Dynamics?5 May 2026
- 2026Frictive Policy Optimization for LLMs: Epistemic Intervention, Risk-Sensitive Control, and Reflective Alignment28 Apr 2026
- 2026The LLM Fallacy: Misattribution in AI-Assisted Cognitive Workflows16 Apr 2026
- 2026Saying More Than They Know: A Framework for Quantifying Epistemic-Rhetorical Miscalibration in Large Language Models27 Mar 2026
- 2026Epistemic Observability in Language Models20 Mar 2026
- 2026From Sycophancy to Sensemaking: Premise Governance for Human-AI Decision Making2 Feb 2026
- 2025Beyond One-Way Influence: Bidirectional Opinion Dynamics in Multi-Turn Human-LLM Interactions22 Oct 2025
- 2025Matching Game Preferences Through Dialogical Large Language Models: A Perspective26 Jul 2025
Similar, not cited
Close in wording, no citation either way
- 2025Decentralising LLM Alignment: A Case for Context, Pluralism, and Participation
- 2026Architecting Trust in Artificial Epistemic Agents
- 2026On Epistemic Diversity in Large Language Models
- 2025The Epistemic Suite: A Post-Foundational Diagnostic Methodology for Assessing AI Knowledge Claims
- 2026Ethical LLM-Assisted Research: A Framework for Responsible Delegation, Verification, and Epistemic Value
- 2026Co-Constructing Alignment: A Participatory Approach to Situate AI Values ↗
- 2025Community-Aligned Behavior Under Uncertainty: Evidence of Epistemic Stance Transfer in LLMs
- 2025A Good Plan is Hard to Find: Aligning Models with Preferences is Misaligned with What Helps Users