mlsanityData to 14 Oct 2026

Diagnosing and Mitigating Sycophancy and Skepticism in LLM Causal Judgment

arXiv 2601.08258 ↗First posted 13 Jan 2026cs.AIcited by 5

On the map: LLM reasoning → Causal reasoning in LLMs

Abstract (arXiv)

Large language models increasingly fail in a way that scalar accuracy cannot diagnose: they produce a sound reasoning trace and then abandon it under social pressure or an authoritative hint. We argue that this is a control failure, not a knowledge failure, and that it requires an evaluation surface richer than a single accuracy number. We introduce CAUSALT3, a 454 instance expert curated benchmark for causal reasoning across all three rungs of Pearl's ladder, and a three axis evaluation that decomposes performance into Utility (sensitivity to valid causal claims), Safety (specificity against invalid ones), and Wise Refusal (calibrated abstention on genuinely underdetermined items). On this surface we document three reproducible pathologies: a Skepticism Trap at L1 where capable models over refuse sound links, a Sycophancy Trap at L2 where confident user pressure flips correct answers, and a Scaling Paradox at L3 where a frontier model underperforms an older one on counterfactual Safety by 55 points. To mitigate these failures without retraining, we propose Regulated Causal Anchoring (RCA), an inference time process verifier that audits trace output consistency under a PID style feedback loop and abstains rather than ratifying a detected mismatch. Across CAUSALT3 and a supporting CAP-GSM8K stress test, RCA reduces sycophantic acceptance to near zero while preserving valid hint acceptance, recasting trustworthy reasoning as a question of inference time control rather than scale.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Causal reasoning in LLMs, one of 17 directions in LLM reasoning. Builds on 17 of its 50 references that are in the map. First posted on arXiv 13 Jan 2026.

Builds on

17 of 50 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causality28 Apr
  2. 2022e-CARE: a New Dataset for Exploring Explainable Causal Reasoning12 May

Other references · 15

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  4. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  5. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  6. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  7. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  8. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 407
  9. 2023Inverse Scaling: When Bigger Isn't Bettercited by 207
  10. 2023Causal Parrots: Large Language Models May Talk Causality But Are Not Causalcited by 150
  11. 2023Can Large Language Models Infer Causation from Correlation?cited by 146
  12. 2025Measuring Sycophancy of Language Models in Multi-turn Dialoguescited by 104
3 more
  1. 2023Understanding Causality with Large Language Models: Feasibility and Opportunitiescited by 53
  2. 2025BrokenMath: A Benchmark for Sycophancy in Theorem Proving with LLMscited by 27
  3. 2025Epidemiology of Large Language Models: A Benchmark for Observational Distribution Knowledgecited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Internal Reasoning vs. External Control: A Thermodynamic Analysis of Sycophancy in Large Language Models
  2. 2026RAudit: A Blind Auditing Protocol for Large Language Model Reasoning
  3. 2024A Critical Review of Causal Reasoning Benchmarks for Large Language Models
  4. 2025Reasoning Isn't Enough: Examining Truth-Bias and Sycophancy in LLMs
  5. 2026CausalFlip: A Benchmark for LLM Causal Judgment Beyond Semantic Matching
  6. 2026Uncovering Hidden Correctness in LLM Causal Reasoning via Symbolic Verification
  7. 2026Do LLMs Share Human-Like Biases? Causal Reasoning Under Prior Knowledge, Irrelevant Context, and Varying Compute Budgets
  8. 2026Causal Tongue-Tie: LLMs Can Encode Causal Direction, But Their Yes/No Outputs Fail to Express