Seeing Through the Forgery Is Not Enough: Regressive Sycophancy under Fabricated Evidence in Medical Vision-Language Models
On the map: Multimodal LLMs / vision-language models → Medical vision-language models
Abstract (arXiv)
A medical vision-language model may analyze a given CT scan and correctly detect both kidneys in view. Continuing this conversation, if a user provides a prescription with an opposite direction, medical VLMs often sycophantically choose to realign their diagnosis. In this work, we study regressive sycophancy under fabricated evidence, where a model abandons a correct answer when pushback comes with a fake document. We test four open VLMs on 400 VQA-RAD yes/no questions in 19,200 multi-turn conversations, with and without a fabricated prescription. Without a disclosure that documents are synthetic, the models abandon 80.1% of 6,488 correct answers. Notably, under one script each, the same prescription raises the flip rate of Qwen2.5-VL-7B from 45.9% to 89.6% and lowers that of Qwen3-VL-8B from 77.7% to 50.9%. A model rejected the prescription while still preserving the correct answer in 915 of 3,232 conversations with the prescription (28.3%), and 40% of these flipped in later turns. System-prompt sentences that disclose the documents as synthetic lower one model's flip rate from 77.8% to 47.3%. Thus, we conclude that awareness about a forgery does not prevent a model from adopting its answer.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 22 of its 56 references that are in the map. First posted on arXiv 13 Oct 2026.
Builds on
22 of 56 references.
Closest ancestors · 1
Same direction, newest first
Other references · 21
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
- 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
- 2025Measuring Sycophancy of Language Models in Multi-turn Dialoguescited by 104
- 2023Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experimentcited by 77
- 2025The Hawthorne Effect in Reasoning Models: Evaluating and Steering Test Awarenesscited by 30
- 2026LLMs Know They're Wrong and Agree Anyway: The Shared Sycophancy-Lying Circuitcited by 10
- 2026When Correct Beliefs Collapse: Epistemic Resilience of LLMs under Clinical Pressurecited by 6
- 2026LURE: Live-Usage Replay Evaluations for Reducing Evaluation Awarenesscited by 4
- 2026To See or To Please: Uncovering Visual Sycophancy and Split Beliefs in VLMscited by 3
- 2026Evaluation Awareness Is Not One Capability: Evidence from Open Language Modelscited by 3
- 2026Why LLMs Give In: Conversational Factors and Reasoning Behind Medical Sycophancycited by 3
9 more
- 2026SPINE: Measuring LLM Sycophancy under Sustained Multi-Turn Pressurecited by 3
- 2026Sycophancy Undermines Epistemic Vigilance in Cooperative Vision-Language Taskscited by 2
- 2026MedPRESS: A Multi-turn Benchmark for Patient-Pressure-Induced Medical Sycophancy in LLMscited by 2
- 2026Same Semantics, Different Outcome: On the Modality Robustness of Multimodal LLMs under Knowledge Conflictcited by 2
- 2026The Chain Holds, the Answer Folds: Trace-Answer Dissociation in Reasoning Models Under Adversarial Pressurecited by 1
- 2026Looking Again: Measuring Sycophancy in the Reasoning Chains of Multimodal Models Under Pressurecited by 1
- 2026Do Multimodal LLMs See Before They Read? Diagnosing Contextual Sycophancycited by 1
- 2026GradeTrap: Authority Cues in Images Shift VLM Judgments Despite Explicit Instructions to Ignore Themcited by 1
- 2026Seeing is not Enough: Vision-Language Models Perceive Evidence but Fail to Actcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidence
- 2026To Agree or To Be Right? The Grounding-Sycophancy Tradeoff in Medical Vision-Language Models
- 2025EchoBench: Benchmarking Sycophancy in Medical Large Vision-Language Models
- 2025Benchmarking and Mitigating Sycophancy in Medical Vision Language Models
- 2026Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigation
- 2026Toward Guarantees for Clinical Reasoning in Vision Language Models via Formal Verification
- 2026Text Over Image: Auditing Multimodal Robustness in Synthetic Medical Image Detection
- 2026Knowing When Not to Answer: Abstention and Refusal Reasoning in Vision--Language Models