mlsanityData to 7 Oct 2026

Epistemic Alignment: A Mediating Framework for User-LLM Knowledge Delivery

arXiv 2504.01205 ↗First posted 1 Apr 2025cs.HC cs.AI cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

LLMs increasingly serve as tools for knowledge acquisition, yet users cannot effectively specify how they want information presented. When users request that LLMs "cite reputable sources," "express appropriate uncertainty," or "include multiple perspectives," they discover that current interfaces provide no structured way to articulate these preferences. The result is prompt sharing folklore: community-specific copied prompts passed through trust relationships rather than based on measured efficacy. We propose the Epistemic Alignment Framework, a set of ten challenges in knowledge transmission derived from the philosophical literature of epistemology, concerning issues such as evidence quality assessment and calibration of testimonial reliance. The framework serves as a structured intermediary between user needs and system capabilities, creating a common vocabulary to bridge the gap between what users want and what systems deliver. Through a thematic analysis of custom prompts and personalization strategies shared on online communities where these issues are actively discussed, we find users develop elaborate workarounds to address each of the challenges. We then apply our framework to two prominent model providers, OpenAI and Anthropic, through content analysis of their documented policies and product features. Our analysis shows that while these providers have partially addressed the challenges we identified, they fail to establish adequate mechanisms for specifying epistemic preferences, lack transparency about how preferences are implemented, and offer no verification tools to confirm whether preferences were followed. For AI developers, the Epistemic Alignment Framework offers concrete guidance for supporting diverse approaches to knowledge; for users, it works toward information delivery that aligns with their specific needs rather than defaulting to one-size-fits-all approaches.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 34 of its 64 references that are in the map. First posted on arXiv 1 Apr 2025.

Builds on

34 of 64 references.

References · 34

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  3. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  4. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  5. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  6. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
  7. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  8. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  9. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  10. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  11. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  12. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
22 more
  1. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  2. 2024A Roadmap to Pluralistic Alignmentcited by 175
  3. 2024Fine-grained Hallucination Detection and Editing for Language Modelscited by 146
  4. 2024Language Models with Conformal Factuality Guaranteescited by 128
  5. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 124
  6. 2024Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaborationcited by 111
  7. 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 107
  8. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 100
  9. 2024Personalization of Large Language Models: A Surveycited by 100
  10. 2024Tug-of-War Between Knowledge: Exploring and Resolving Knowledge Conflicts in Retrieval-Augmented Language Modelscited by 93
  11. 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 85
  12. 2024Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Datacited by 65
  13. 2023The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensivenesscited by 63
  14. 2023Resolving Knowledge Conflicts in Large Language Modelscited by 49
  15. 2024Can AI Assistants Know What They Don't Know?cited by 47
  16. 2024NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Modelscited by 47
  17. 2024A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Taskscited by 46
  18. 2023Citation: A Key to Building Responsible and Accountable Large Language Modelscited by 20
  19. 2024User Intent Recognition and Satisfaction with Large Language Models: A User Study with ChatGPTcited by 19
  20. 2025Citations and Trust in LLM Generated Responsescited by 17
  21. 2024DebateQA: Evaluating Question Answering on Debatable Knowledgecited by 12
  22. 2024Generative AI in Education: A Study of Educators' Awareness, Sentiments, and Influencing Factorscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 10.

14 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2025Decentralising LLM Alignment: A Case for Context, Pluralism, and Participation
  2. 2026Architecting Trust in Artificial Epistemic Agents
  3. 2026On Epistemic Diversity in Large Language Models
  4. 2025The Epistemic Suite: A Post-Foundational Diagnostic Methodology for Assessing AI Knowledge Claims
  5. 2026Ethical LLM-Assisted Research: A Framework for Responsible Delegation, Verification, and Epistemic Value
  6. 2026Co-Constructing Alignment: A Participatory Approach to Situate AI Values ↗
  7. 2025Community-Aligned Behavior Under Uncertainty: Evidence of Epistemic Stance Transfer in LLMs
  8. 2025A Good Plan is Hard to Find: Aligning Models with Preferences is Misaligned with What Helps Users