On the Robustness of Temporal Vision-Language Models for Surgical Endoscopy Videos
On the map: Multimodal LLMs / vision-language models → Medical vision-language models
Abstract (arXiv)
Temporal vision-language models (TVLMs) offer a reusable, prompt-based interface for surgical video understanding, yet, their robustness under clinically realistic acquisition artifacts in endoscopy remains insufficiently characterized. In practice, degradations such as defocus, haze, motion blur, noise, cautery smoke, and packet loss introduce structured distribution shifts which may compromise video-text alignment. We study the robustness of temporal VLMs under such shifts caused by corruptions in clip frames. We introduce Endo-C6, a compact corruption benchmark of six endoscopy-realistic perturbations evaluated at a fixed high severity, and apply it to public Gastrointestinal (GI) endoscopy and laparoscopic cholecystectomy videos. Under a standardized prompt protocol, we benchmark 3 recent surgical TVLM baselines and analyze robustness in both mean and worst-case settings, spanning 294 dataset-level evaluations. Finally, we present RobustEndoCLIP, obtained by few-shot parameter-efficient tuning with VeRA, outperforming existing TVLM baselines. Our findings show that off-the-shelf TVLMs can exhibit severe worst-case collapse under endoscopy-specific corruptions, whereas lightweight few-shot adaptation can substantially improve corrupted performance and robustness without changing the prompt-based interface. We expect Endo-C6 to support standardized robustness reporting and promote more reliable clinical vision-language systems.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 25 references that are in the map. First posted on arXiv 14 Aug 2026.
Builds on
14 of 25 references.
Closest ancestors · 1
Same direction, newest first
Other references · 13
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
- 2023VeRA: Vector-based Random Matrix Adaptationcited by 300
- 2023Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lecturescited by 68
- 2024HecVL: Hierarchical Video-Language Pretraining for Zero-shot Surgical Phase Recognitioncited by 42
- 2023Challenges in Multi-centric Generalization: Phase and Step Recognition in Roux-en-Y Gastric Bypass Surgerycited by 40
- 2024Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentationcited by 39
- 2019A deep learning framework for quality assessment and restoration in video endoscopycited by 24
- 2024LighTDiff: Surgical Endoscopic Image Low-Light Enhancement with T-Diffusioncited by 12
- 2024VidLPRO: A underlineVideo-underlineLanguage underlinePre-training Framework for underlineRobotic and Laparoscopic Surgerycited by 12
- 2025Adaptation of Multi-modal Representation Models for Multi-task Surgical Computer Visioncited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Challenging Vision-Language Models with Surgical Data: A New Dataset and Broad Benchmarking Study
- 2026An Approach to Enriching Surgical Video Datasets for Fine-Grained Spatial-Temporal Understanding of Vision-Language Models
- 2026EndoVLM: An Endoscopy Vision-Language Pre-training Model via Anatomy-Guided Sparsity and Progressive Alignment
- 2026Scaling Video Pretraining for Surgical Foundation Models
- 2026A report-grounded vision-language foundation model for colonoscopy from 280000 routine reports
- 2025SurgLaVi: Large-Scale Hierarchical Dataset for Surgical Vision-Language Representation Learning
- 2026HeiCo-FOCUS: A Clinically Grounded Dataset for Long-Context Video Understanding
- 2025SurgViVQA: Temporally-Grounded Video Question Answering for Surgical Scene Understanding