mlsanityData to 7 Oct 2026

Can LLM-Generated Text Empower Surgical Vision-Language Pre-training?

arXiv 2604.18134 ↗First posted 20 Apr 2026cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Recent advancements in self-supervised learning have led to powerful surgical vision encoders capable of spatiotemporal understanding. However, extending these visual foundations to multi-modal reasoning tasks is severely bottlenecked by the prohibitive cost of expert textual annotations. To overcome this scalability limitation, we introduce LIME, a large-scale multi-modal dataset derived from open-access surgical videos using human-free, Large Language Model (LLM)-generated narratives. While LIME offers immense scalability, unverified generated texts may contain errors, including hallucinations, that could potentially lead to catastrophically degraded pre-trained medical priors in standard contrastive pipelines. To mitigate this, we propose SurgLIME, a parameter-efficient Vision-Language Pre-training (VLP) framework designed to learn reliable cross-modal alignments using noisy narratives. SurgLIME preserves foundational medical priors using a LoRA-adapted dual-encoder architecture and introduces an automated confidence estimation mechanism that dynamically down-weights uncertain text during contrastive alignment. Evaluations on the AutoLaparo and Cholec80 benchmarks show that SurgLIME achieves competitive zero-shot cross-modal alignment while preserving the robust linear probing performance of the visual foundation model. Dataset, code, and models are publicly available at https://github.com/visurg-ai/SurgLIME.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 41 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

29 of 41 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025MedGRPO: Multi-Task Reinforcement Learning for Heterogeneous Medical Video Understanding6 Dec
  2. 2025SurgLaVi: Large-Scale Hierarchical Dataset for Surgical Vision-Language Representation Learning9 Sep

Other references · 27

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  10. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  11. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  12. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
15 more
  1. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  2. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  3. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  4. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
  5. 2020TransNet V2: An effective deep network architecture for fast shot transition detectioncited by 166
  6. 2022AutoLaparo: A New Dataset of Integrated Multi-tasks for Image-guided Surgical Automation in Laparoscopic Hysterectomycited by 85
  7. 2023Foundation Model for Endoscopy Video Analysis via Large-scale Self-supervised Pre-traincited by 75
  8. 2023Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lecturescited by 68
  9. 2025Scaling up self-supervised learning for improved surgical foundation modelscited by 25
  10. 2026Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoringcited by 21
  11. 2025LEMON: A Large Endoscopic MONocular Dataset and Foundation Model for Perception in Surgical Settingscited by 19
  12. 2025Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoningcited by 14
  13. 2025Not All Errors Are Created Equal: ASCoT Addresses Late-Stage Fragility in Efficient LLM Reasoningcited by 11
  14. 2025Streamline pathology foundation model by cross-magnification distillationcited by 3
  15. 2026SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026SCALPEL: Semantic Cross-modal Alignment via LLM-Powered Encoder Learning for Medical Vision-Language Representation
  2. 2025SurgVLM: A Large Vision-Language Model and Systematic Evaluation Benchmark for Surgical Intelligence
  3. 2024LLaVA-Surg: Towards Multimodal Surgical Assistant via Structured Surgical Video Learning
  4. 2025More performant and scalable: Rethinking contrastive vision-language pre-training of radiology in the LLM era
  5. 2026An Approach to Enriching Surgical Video Datasets for Fine-Grained Spatial-Temporal Understanding of Vision-Language Models
  6. 2024ExGra-Med: Extended Context Graph Alignment for Medical Vision-Language Models
  7. 2024Surgical-LLaVA: Toward Surgical Scenario Understanding via Large Language and Vision Models
  8. 2023Med-UniC: Unifying Cross-Lingual Medical Vision-Language Pre-Training by Diminishing Bias