TextSLIP: Text Self-Supervised CLIP for Medical Report Generation
On the map: Multimodal LLMs / vision-language models → Medical vision-language models
Abstract (arXiv)
Automating radiology report generation is important for improving reporting consistency and clinical workflows . While Contrastive Language--Image Pretraining (CLIP) has advanced medical vision language modeling, existing CLIP-style approaches may still provide insufficient fine-grained semantic supervision for complex report generation. Standard CLIP primarily optimizes cross-modal alignment, without explicitly structuring the textual embedding space that guides visual representation learning. To address this limitation, we propose TextSLIP, a general medical vision-language pretraining framework that augments CLIP with intra-modal text contrastive learning. By improving textual embedding discriminability through self-supervised augmented text pairs, TextSLIP is designed to provide finer-grained linguistic supervision to the visual encoder. As an initial validation, we pretrain TextSLIP on a curated dataset of 7 million brain MRI image-text pairs and fine-tune the pretrained visual encoder within a report generation architecture. In controlled comparisons with CLIP-style baselines, TextSLIP shows consistent improvements on report generation metrics. Ablation studies further suggest that text-side self-supervision contributes to the observed gains. These results indicate that text-level contrastive learning is a promising direction for improving medical visual-textual alignment, while broader validation across additional medical domains remains an important next step.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 20 references that are in the map. First posted on arXiv 24 Jul 2026.
Builds on
14 of 20 references.
Closest ancestors · 2
Same direction, newest first
- 2024UniMed-CLIP: Towards a Unified Image-Text Pretraining Paradigm for Diverse Medical Imaging Modalities13 Dec
- 2024MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicine6 Aug
Other references · 12
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 520
- 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 436
- 2020Generating Radiology Reports via Memory-driven Transformercited by 368
- 2023Demystifying CLIP Datacited by 270
- 2022Cross-modal Memory Networks for Radiology Report Generationcited by 185
- 2023Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generationcited by 118
- 2023METransformer: Radiology Report Generation by Transformer with Multiple Learnable Expert Tokenscited by 101
- 2021ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embeddingcited by 74
- 2024CXPMRG-Bench: Pre-training and Benchmarking for X-ray Medical Report Generation on CheXpert Plus Datasetcited by 35
- 2024AutoRG-Brain: Grounded Report Generation for Brain MRIcited by 24
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025QwenCLIP: Boosting Medical Vision-Language Pretraining via LLM Embeddings and Prompt tuning
- 2025UniCrossAdapter: Multimodal Adaptation of CLIP for Radiology Report Generation
- 2026Learning to See Locally and Align Clinically with Pathology Semantics for Radiology Report Generation
- 2023Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime
- 2025More performant and scalable: Rethinking contrastive vision-language pre-training of radiology in the LLM era
- 2024SERPENT-VLM : Self-Refining Radiology Report Generation Using Vision Language Models
- 2026MedTri: A Platform for Structured Medical Report Normalization to Enhance Vision-Language Pretraining
- 2025SpeechCT-CLIP: Distilling Text-Image Knowledge to Speech for Voice-Native Multimodal CT Analysis