mlsanityData to 14 Oct 2026

OR-VSKC: Resolving Visual-Semantic Knowledge Conflicts in Operating Rooms with Synthetic Data-Guided Alignment

arXiv 2506.22500 ↗First posted 25 Jun 2025cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

also in 1 more
Abstract (arXiv)

Automated identification of surgical safety risks is critical for improving patient outcomes; however, Multimodal Large Language Models (MLLMs) frequently suffer from Visual-Semantic Knowledge Conflicts (VS-KC), a phenomenon where models possess safety knowledge but fail to activate it during visual inspection. Investigating this alignment gap in operating rooms (ORs) is impeded by a critical bottleneck: the scarcity and privacy constraints of real-world OR data depicting safety violations. To address this, we introduce OR-VSKC, a benchmark for studying VS-KC and surgical risk perception in strictly regulated OR environments. Constructed via our Protocol-to-Pixel Generative Framework, OR-VSKC comprises 28,190 high-fidelity synthetic images grounded in authoritative safety standards, complemented by a 713-image expert-authored challenge subset validated by multiple experts. The full benchmark is built from authentic OR contexts drawn from the 4D-OR and CAMMA-MVOR datasets, where the 4D-OR-based portion serves as the primary benchmark core and the CAMMA-MVOR-based portion is reserved for external validation and cross-dataset generalization analysis. Evaluations of state-of-the-art MLLMs reveal substantial reliability gaps even in advanced generalist models. Furthermore, experiments show that fine-tuning on OR-VSKC effectively mitigates VS-KC and enables robust generalization to unseen camera viewpoints. We open-source the code and dataset to support reproducible research in safety-critical medical environments. The source code is available at https://github.com/zgg2577/VS-KC.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 39 references that are in the map. First posted on arXiv 25 Jun 2025.

Builds on

28 of 39 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicine14 May
  2. 2024ORacle: Large Vision-Language Models for Knowledge-Guided Holistic OR Domain Modeling10 Apr

Other references · 26

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  6. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  7. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  8. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 449
  9. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
  10. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 376
  11. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
  12. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 364
14 more
  1. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 290
  2. 2024Knowledge Conflicts for LLMs: A Surveycited by 283
  3. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 278
  4. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 202
  5. 2023Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Modelscited by 118
  6. 2018MVOR: A Multi-view RGB-D Operating Room Dataset for 2D and 3D Human Pose Estimationcited by 49
  7. 20224D-OR: Semantic Scene Graphs for OR Domain Modelingcited by 49
  8. 2023Unveiling the Pitfalls of Knowledge Editing for Large Language Modelscited by 43
  9. 2022Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inferencecited by 40
  10. 2024Gemini Goes to Med School: Exploring the Capabilities of Multimodal Large Language Models on Medical Challenge Problems & Hallucinationscited by 27
  11. 2024PitVQA: Image-grounded Text Embedding LLM for Visual Question Answering in Pituitary Surgerycited by 23
  12. 2023A Causal View of Entity Bias in (Large) Language Modelscited by 21
  13. 2022Does Your Model Classify Entities Reasonably? Diagnosing and Mitigating Spurious Correlations in Entity Typingcited by 16
  14. 2024SuFIA: Language-Guided Augmented Dexterity for Robotic Surgical Assistantscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Consistent but Dangerous: Per-Sample Safety Classification Reveals False Reliability in Medical Vision-Language Models
  2. 2025SurgVLM: A Large Vision-Language Model and Systematic Evaluation Benchmark for Surgical Intelligence
  3. 2025Systematic Evaluation of Large Vision-Language Models for Surgical Artificial Intelligence
  4. 2026EviSafe: Evidence-Grounded Safety Evaluation for Vision-Language Models
  5. 2025VSCBench: Bridging the Gap in Vision-Language Model Safety Calibration
  6. 2025VLSU: Mapping the Limits of Joint Multimodal Understanding for AI Safety
  7. 2026Toward Autonomous Laboratory Safety Monitoring with Vision Language Models: Learning to See Hazards Through Scene Structure
  8. 2026MedObvious: Exposing the Medical Moravec's Paradox in VLMs via Clinical Triage