mlsanityData to 7 Oct 2026

Robust for the Wrong Reasons: The Representational Geometry of LLM Robustness to Science Skepticism

arXiv 2607.01951 ↗First posted 2 Jul 2026physics.soc-ph cs.AI cs.CLcited by 1

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large language models (LLMs) are increasingly consulted on contested scientific questions, raising the concern that they will sycophantically retreat from established consensus when a user signals doubt -- drifting toward a false balance that treats settled science as one view among several. We test this across three open instruction-tuned models (Llama-3.1-8B, Qwen2.5-7B, Mistral-7B), three consensus-science domains (climate, vaccines, evolution), and single- and multi-turn settings, combining behavioral measurement with linear probing and activation patching. We do not observe sycophantic retreat. Instead, models show three distinct policies under the same skeptical pressure: reactive assertion, where consensus assertion increases rather than decreases (Llama); surface hedging, where tone softens while the position holds (Qwen); and non-response (Mistral). Pairwise judgments confirm the reactive shift is stance, not style (63.6%, p=.007), and a decomposition identifies increased consensus assertion, not false balance, as its driver (beta=+0.042 per dose, p<1e-77). Linear probes localize the divergence to middle layers -- perfect separation in Llama and Qwen versus 72% in Mistral, with non-overlapping confidence intervals -- indicating the non-responsive model does not linearly represent the skepticism signal at all. Crucially, this robustness does not transfer: it attenuates across domains and, in the safety-critical vaccine domain, can reverse, with myth-rebuttal weakening under skeptical pressure. We synthesize these into a four-way taxonomy separating active from accidental robustness, and argue that behavioral evaluation alone cannot distinguish a model that resists skepticism because it understands the signal from one that only appears to resist because it fails to perceive it.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 10 of its 24 references that are in the map. First posted on arXiv 2 Jul 2026.

Builds on

10 of 24 references.

References · 10

  1. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  2. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  3. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  4. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  5. 2023Steering Language Models With Activation Engineeringcited by 824
  6. 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
  7. 2020CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claimscited by 195
  8. 2023Simple synthetic data reduces sycophancy in large language modelscited by 174
  9. 2025Measuring Sycophancy of Language Models in Multi-turn Dialoguescited by 91
  10. 2023When Large Language Models contradict humans? Large Language Models' Sycophantic Behaviourcited by 61

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025Sycophancy under Pressure: Evaluating and Mitigating Sycophantic Bias via Adversarial Dialogues in Scientific QA
  2. 2026Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debate
  3. 2026Evaluating Evidence Grounding Under User Pressure in Instruction-Tuned Language Models
  4. 2026Diagnosing and Mitigating Sycophancy and Skepticism in LLM Causal Judgment
  5. 2026TRACES: A Benchmark for Epistemic Reliability in Scientific Reasoning by LLMs
  6. 2025Community-Aligned Behavior Under Uncertainty: Evidence of Epistemic Stance Transfer in LLMs
  7. 2025SciTrust 2.0: A Comprehensive Framework for Evaluating Trustworthiness of Large Language Models in Scientific Applications
  8. 2025Single LLM Debate, MoLaCE: Mixture of Latent Concept Experts Against Confirmation Bias