mlsanityData to 7 Oct 2026

Med-UniC: Unifying Cross-Lingual Medical Vision-Language Pre-Training by Diminishing Bias

arXiv 2305.19894 ↗First posted 31 May 2023cs.CL cs.AI cs.CVcited by 57

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

The scarcity of data presents a critical obstacle to the efficacy of medical visionlanguage pre-training (VLP). A potential solution lies in the combination of datasets from various language communities. Nevertheless, the main challenge stems from the complexity of integrating diverse syntax and semantics, language-specific medical terminology, and culture-specific implicit knowledge. Therefore, one crucial aspect to consider is the presence of community bias caused by different languages. This paper presents a novel framework named Unifying Cross-Lingual Medical Vision-Language Pre-Training (Med-UniC), designed to integrate multimodal medical data from the two most prevalent languages, English and Spanish. Specifically, we propose Cross-lingual Text Alignment Regularization (CTR) to explicitly unify cross-lingual semantic representations of medical reports originating from diverse language communities. CTR is optimized through latent language disentanglement, rendering our optimization objective to not depend on negative samples, thereby significantly mitigating the bias from determining positive-negative sample pairs within analogous medical reports. Furthermore, it ensures that the cross-lingual representation is not biased toward any specific language community. Med-UniC reaches superior performance across 5 medical image tasks and 10 datasets encompassing over 30 diseases, offering a versatile framework for unifying multi-modal medical data within diverse linguistic communities. The experimental outcomes highlight the presence of community bias in cross-lingual VLP. Reducing this bias enhances the performance not only in vision-language tasks but also in uni-modal visual tasks.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 57 references that are in the map. First posted on arXiv 31 May 2023.

Builds on

36 of 57 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2023GPT-4 Technical Reportcited by 18,132
  7. 2023Segment Anythingcited by 8,158
  8. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  9. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  11. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  12. 2018YOLOv3: An Incremental Improvementcited by 2,652
24 more
  1. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  2. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  3. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  4. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  5. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
  6. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  7. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 434
  8. 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 336
  9. 2019PadChest: A large chest x-ray image dataset with multi-label annotated reportscited by 308
  10. 2020COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-Ray Imagescited by 264
  11. 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
  12. 2022Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation Learningcited by 159
  13. 2023MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training in Radiologycited by 149
  14. 2022CyCLIP: Cyclic Contrastive Language-Image Pretrainingcited by 127
  15. 2023Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applicationscited by 121
  16. 2023Advancing Radiograph Representation Learning with Masked Record Modelingcited by 75
  17. 2023Machine learning with data assimilation and uncertainty quantification for dynamical systems: a reviewcited by 74
  18. 2021UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-trainingcited by 64
  19. 2023Frozen Language Model Helps ECG Zero-Shot Learningcited by 62
  20. 2022Vision-and-Language Pretrained Models: A Surveycited by 50
  21. 2022Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-trainingcited by 24
  22. 2022G-MAP: General Memory-Augmented Pre-trained Language Model for Domain Taskscited by 9
  23. 2022Spatio-Temporal Contrastive Learning Enhanced GNNs for Session-based Recommendationcited by 8
  24. 2021S-SimCSE: Sampled Sub-networks for Contrastive Learning of Sentence Embeddingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 13; 2024: 22; 2025: 15; 2026: 7.

57 papers.

Sort:

By area:

  1. 2023Efficient Large Language Models: A Surveycited by 183
  2. 2024Exploring scalable medical image encoders beyond text supervisioncited by 129
  3. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  4. 2024LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inferencecited by 93
  5. 2024Detecting Multimedia Generated by Large AI Models: A Surveycited by 73
  6. 2024Zero-Shot ECG Classification with Multimodal Learning and Test-time Clinical Knowledge Enhancementcited by 66
  7. 2023Generative Text-Guided 3D Vision-Language Pretraining for Unified Medical Image Segmentationcited by 47
  8. 2023M-FLAG: Medical Vision-Language Pre-training with Frozen Language Models and Latent Space Geometry Optimizationcited by 43
  9. 2023CLIP in Medical Imaging: A Surveycited by 40
  10. 2023Enhancing Representation in Radiography-Reports Foundation Model: A Granular Alignment Algorithm Using Masked Contrastive Learningcited by 34
  11. 2024D2O: Dynamic Discriminative Operations for Efficient Long-Context Inference of Large Language Modelscited by 30
  12. 2024MEIT: Multimodal Electrocardiogram Instruction Tuning on Large Language Models for Report Generationcited by 29
38 more
  1. 2023ETP: Learning Transferable ECG Representations via ECG-Text Pre-trainingcited by 28
  2. 2023Fine-Grained Image-Text Alignment in Medical Imaging Enables Explainable Cyclic Image-Report Generationcited by 25
  3. 2024ECG Semantic Integrator (ESI): A Foundation ECG Model Pretrained with LLM-Enhanced Cardiological Textcited by 25
  4. 2023IMITATE: Clinical Prior Guided Hierarchical Vision-Language Pre-trainingcited by 23
  5. 2024MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimizationcited by 21
  6. 2024Has Multimodal Learning Delivered Universal Intelligence in Healthcare? A Comprehensive Surveycited by 20
  7. 2023Learning Multiscale Consistency for Self-supervised Electron Microscopy Instance Segmentationcited by 16
  8. 2023Utilizing Synthetic Data for Medical Vision-Language Pre-training: Bypassing the Need for Real Imagescited by 16
  9. 2023T3D: Advancing 3D Medical Vision-Language Pre-training by Learning Multi-View Visual Consistencycited by 14
  10. 2024Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-trainingcited by 13
  11. 2023G2D: From Global to Dense Radiography Representation Learning via Vision-Language Pre-trainingcited by 12
  12. 2024Open Challenges and Opportunities in Federated Foundation Models Towards Biomedical Healthcarecited by 12
  13. 2025Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metricscited by 12
  14. 2024A Survey of Medical Vision-and-Language Applications and Their Techniquescited by 11
  15. 2024Multi-View and Multi-Scale Alignment for Contrastive Language-Image Pre-training in Mammographycited by 9
  16. 2025Foundation Models in Medical Imaging: A Review and Outlookcited by 7
  17. 2024MLIP: Efficient Multi-Perspective Language-Image Pretraining with Exhaustive Data Utilizationcited by 6
  18. 2025SuPreME: A Supervised Pre-training Framework for Multimodal ECG Representation Learningcited by 6
  19. 2025Towards All-in-One Medical Image Re-Identificationcited by 6
  20. 2024Enhancing the vision-language foundation model with key semantic knowledge-emphasized report refinementcited by 5
  21. 2025Knowledge-enhanced Multimodal ECG Representation Learning with Arbitrary-Lead Inputscited by 5
  22. 2023Breast Cancer Classification with Enhanced Interpretability: DALAResNet50 and DT Grad-CAMcited by 4
  23. 2024Enhancing medical vision-language contrastive learning via inter-matching relation modellingcited by 4
  24. 2025A Narrative Review on Large AI Models in Lung Cancer Screening, Diagnosis, and Treatment Planningcited by 4
  25. 2024Enhancing Representation in Medical Vision-Language Foundation Models via Multi-Scale Information Extraction Techniquescited by 3
  26. 2024How Does Diverse Interpretability of Textual Prompts Impact Medical Vision-Language Zero-Shot Tasks?cited by 2
  27. 2025Semantics versus Identity: A Divide-and-Conquer Approach towards Adjustable Medical Image De-Identificationcited by 2
  28. 2025MedPatch: Confidence-Guided Multi-Stage Fusion for Multimodal Clinical Datacited by 2
  29. 2024MeDSLIP: Medical Dual-Stream Language-Image Pre-training with Pathology-Anatomy Semantic Alignmentcited by 1
  30. 2024Universal Medical Image Representation Learning with Compositional Decoderscited by 1
  31. 2025Meta-Entity Driven Triplet Mining for Aligning Medical Vision-Language Modelscited by 1
  32. 2025Position: Restructuring of Categories and Implementation of Guidelines Essential for VLM Adoption in Healthcarecited by 1
  33. 2025Improving Medical Visual Representation Learning with Pathological-level Cross-Modal Alignment and Correlation Explorationcited by 1
  34. 2026SemEnrich: Self-Supervised Semantic Enrichment of Radiology Reports for Vision-Language Learningcited by 1
  35. 2026ASAP: Advancing Medical Volumetric Representation Learning with Anatomy-aware Semantically-adaptive Pre-trainingcited by 1
  36. 2026CAP: Towards PPG Universal Representation Learning with Patient-level Supervisioncited by 1
  37. 2024Self-Supervised Learning for Building Robust Pediatric Chest X-ray Classification Modelscited by 0
  38. 2025Group-CLIP Uncertainty Modeling for Group Re-Identificationcited by 0

50 most cited of 57 All 57 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026SCALPEL: Semantic Cross-modal Alignment via LLM-Powered Encoder Learning for Medical Vision-Language Representation
  2. 2026Analyzing and Mitigating Cross-Lingual Degradation in Multilingual Medical VQA
  3. 2026MedTri: A Platform for Structured Medical Report Normalization to Enhance Vision-Language Pretraining
  4. 2024Medical Vision-Language Pre-Training for Brain Abnormalities
  5. 2026MedLayBench-V: A Large-Scale Benchmark for Expert-Lay Semantic Alignment in Medical Vision Language Models
  6. 2023ECAMP: Entity-centered Context-aware Medical Vision Language Pre-training
  7. 2025A Reality Check of Vision-Language Pre-training in Radiology: Have We Progressed Using Text?
  8. 2026Can LLM-Generated Text Empower Surgical Vision-Language Pre-training?