mlsanityData to 7 Oct 2026

Epistemic Stance Flexibility Probing: Measuring Prompt-Conditioned Register Shift in Large Language Models

arXiv 2607.12739 ↗First posted 14 Jul 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

A language model may be asked either what experts believe about a contested claim or what it believes about the claim itself. A trustworthy conversational agent should distinguish these two requests and respond in different epistemic registers: neutral attribution in the first case and stance expression in the second. Whether such a shift occurs-and whether it occurs coherently-is not directly assessed by existing benchmarks for accuracy, instruction following, or safety. We introduce ESFP, a behavioral benchmark that treats the contrast between externally attributed and self-attributed prompts as the fundamental unit of measurement. ESFP consists of 104 carefully controlled items spanning six epistemic categories and five phrasing templates, and evaluates model responses along four complementary dimensions: lexical self-attribution, representation-level responsiveness to role framing, sentence-level stance content density assessed by an LLM judge panel, and cross-condition stance consistency. Evaluating eight frontier models from five vendors, we find that epistemic flexibility is largely orthogonal to general model capability: a 27B open-weight model matches the strongest proprietary systems, the flagship model of one family underperforms its lightweight counterpart, and reasoning-optimized models do not consistently exhibit higher flexibility. Stance content density provides the strongest signal, while surface-level lexical markers such as 'I think' can change substantially without corresponding changes in expressed stance. We provide item-level bootstrap confidence intervals, weight-sensitivity analyses, and an explicit discussion of the interpretation limits of the composite score. ESFP measures a model's propensity to adapt its epistemic stance under changing attribution conditions, rather than a general competence measure.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 26 references that are in the map. First posted on arXiv 14 Jul 2026.

Builds on

17 of 26 references.

References · 17

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  7. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  8. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  9. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  10. 2023Whose Opinions Do Language Models Reflect?cited by 759
  11. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  12. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
5 more
  1. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
  2. 2021Understanding Dataset Difficulty with V-Usable Informationcited by 288
  3. 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
  4. 2026Capabilities Ain't All You Need: Measuring Propensities in AIcited by 6
  5. 2026Measuring Progress Toward AGI: A Cognitive Frameworkcited by 6

Built on by

Not cited yet.