mlsanityData to 14 Oct 2026

Caved or Convinced: Temporal Sampling Gates Claim Deference in Video Large Language Models

arXiv 2608.03160 ↗First posted 4 Aug 2026cs.MM cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

When asked which of two events came first, video large language models can fail in two opposite ways: cave to a false claim, or reject a true one. Prior video sycophancy work measures only the first and mitigates it by teaching the model to trust the user less, a fix known in text and image models to worsen the second. In video, both failures come from two causes the literature treats as one: availability, whether the sparse sampled frames contain the two events, and weighting, whether that evidence is trusted over the user. We separate them with two interventions that keep the claim fixed: a frame-preserving reorder that flips the claim's truth, and a sampling-offset shift that captures or misses both events at a fixed frame budget. When the events are missed, the two twins present identical frames, so each of the nine models we evaluate accepts a true and a false claim at the same rate, making Youden's J=0 by construction. Availability is necessary but not sufficient. Five of the nine read the order, yet four of those five still cave to the false claim, so their deference hits a weighting ceiling. Since trust cannot be calibrated over evidence that was never sampled, we propose a reversal test that cancels the model's order prior by scoring the sampled frames forward and reversed, then answers, resamples, or abstains without reading the claim. The test raises the order accuracy to 0.92-1.00 on the models that read the order and abstains rather than guesses on those that cannot.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 32 of its 43 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

32 of 43 references.

References · 32

  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  2. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  3. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  4. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  5. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  6. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  7. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  8. 2017Selective Classification for Deep Neural Networkscited by 852
  9. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  10. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  11. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 688
  12. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
20 more
  1. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  2. 2020Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlationcited by 425
  3. 2024TempCompass: Do Video LLMs Really Understand Videos?cited by 363
  4. 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 318
  5. 2024Multi-Modal Hallucination Control by Visual Information Groundingcited by 208
  6. 2019Cross-task weakly supervised learning from instructional videoscited by 199
  7. 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Groundingcited by 139
  8. 2024VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Modelscited by 90
  9. 2025Words or Vision: Do Vision-Language Models Have Blind Faith in Text?cited by 89
  10. 2024VidHalluc: Evaluating Temporal Hallucinations in Multimodal Large Language Models for Video Understandingcited by 68
  11. 2023VITATECS: A Diagnostic Dataset for Temporal Concept Understanding of Video-Language Modelscited by 55
  12. 2024From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuningcited by 54
  13. 2025FRAG: Frame Selection Augmented Generation for Long Video and Long Document Understandingcited by 27
  14. 2025Flattery in Motion: Benchmarking and Analyzing Sycophancy in Video-LLMscited by 21
  15. 2024Sycophancy in Vision-Language Models: A Systematic Analysis and an Inference-Time Mitigation Frameworkcited by 17
  16. 2024Have the VLMs Lost Confidence? A Study of Sycophancy in VLMscited by 15
  17. 2025BASIL: Bayesian Assessment of Sycophancy in LLMscited by 13
  18. 2025Sycophancy Mitigation Through Reinforcement Learning with Uncertainty-Aware Adaptive Reasoning Trajectoriescited by 11
  19. 2025SEASON: Mitigating Temporal Hallucination in Video Large Language Models via Self-Diagnostic Contrastive Decodingcited by 10
  20. 2025PARROT: Persuasion and Agreement Robustness Rating of Output Truth -- A Sycophancy Robustness Benchmark for LLMscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.