Can LLM-Generated Text Empower Surgical Vision-Language Pre-training?
On the map: Multimodal LLMs / vision-language models → Medical vision-language models
Abstract (arXiv)
Recent advancements in self-supervised learning have led to powerful surgical vision encoders capable of spatiotemporal understanding. However, extending these visual foundations to multi-modal reasoning tasks is severely bottlenecked by the prohibitive cost of expert textual annotations. To overcome this scalability limitation, we introduce LIME, a large-scale multi-modal dataset derived from open-access surgical videos using human-free, Large Language Model (LLM)-generated narratives. While LIME offers immense scalability, unverified generated texts may contain errors, including hallucinations, that could potentially lead to catastrophically degraded pre-trained medical priors in standard contrastive pipelines. To mitigate this, we propose SurgLIME, a parameter-efficient Vision-Language Pre-training (VLP) framework designed to learn reliable cross-modal alignments using noisy narratives. SurgLIME preserves foundational medical priors using a LoRA-adapted dual-encoder architecture and introduces an automated confidence estimation mechanism that dynamically down-weights uncertain text during contrastive alignment. Evaluations on the AutoLaparo and Cholec80 benchmarks show that SurgLIME achieves competitive zero-shot cross-modal alignment while preserving the robust linear probing performance of the visual foundation model. Dataset, code, and models are publicly available at https://github.com/visurg-ai/SurgLIME.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 41 references that are in the map. First posted on arXiv 20 Apr 2026.
Builds on
29 of 41 references.
Closest ancestors · 2
Same direction, newest first
- 2025MedGRPO: Multi-Task Reinforcement Learning for Heterogeneous Medical Video Understanding6 Dec
- 2025SurgLaVi: Large-Scale Hierarchical Dataset for Surgical Vision-Language Representation Learning9 Sep
Other references · 27
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
- 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
- 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
- 2022Survey of Hallucination in Natural Language Generationcited by 2,056
15 more
- 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
- 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
- 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
- 2020TransNet V2: An effective deep network architecture for fast shot transition detectioncited by 166
- 2022AutoLaparo: A New Dataset of Integrated Multi-tasks for Image-guided Surgical Automation in Laparoscopic Hysterectomycited by 85
- 2023Foundation Model for Endoscopy Video Analysis via Large-scale Self-supervised Pre-traincited by 75
- 2023Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lecturescited by 68
- 2025Scaling up self-supervised learning for improved surgical foundation modelscited by 25
- 2026Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoringcited by 21
- 2025LEMON: A Large Endoscopic MONocular Dataset and Foundation Model for Perception in Surgical Settingscited by 19
- 2025Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoningcited by 14
- 2025Not All Errors Are Created Equal: ASCoT Addresses Late-Stage Fragility in Efficient LLM Reasoningcited by 11
- 2025Streamline pathology foundation model by cross-magnification distillationcited by 3
- 2026SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Modelscited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026SCALPEL: Semantic Cross-modal Alignment via LLM-Powered Encoder Learning for Medical Vision-Language Representation
- 2025SurgVLM: A Large Vision-Language Model and Systematic Evaluation Benchmark for Surgical Intelligence
- 2024LLaVA-Surg: Towards Multimodal Surgical Assistant via Structured Surgical Video Learning
- 2025More performant and scalable: Rethinking contrastive vision-language pre-training of radiology in the LLM era
- 2026An Approach to Enriching Surgical Video Datasets for Fine-Grained Spatial-Temporal Understanding of Vision-Language Models
- 2024ExGra-Med: Extended Context Graph Alignment for Medical Vision-Language Models
- 2024Surgical-LLaVA: Toward Surgical Scenario Understanding via Large Language and Vision Models
- 2023Med-UniC: Unifying Cross-Lingual Medical Vision-Language Pre-Training by Diminishing Bias