# MEDBind: Unifying Language and Multimodal Medical Data Embeddings

arXiv: https://arxiv.org/abs/2403.12894 · HTML: https://mlsanity.com/paper/2403.12894/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 19 of its 32 references that are in the map.
- First posted on arXiv 19 Mar 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Medical vision-language models](https://mlsanity.com/mllm/medical-vision-language-models/)

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [MedCLIP: Contrastive Learning from Unpaired Medical Images and Text](https://mlsanity.com/paper/2210.10163/) (2022)
- [Making the Most of Text Semantics to Improve Biomedical Vision--Language Processing](https://mlsanity.com/paper/2204.09817/) (2022)
- [Towards Generalist Biomedical AI](https://mlsanity.com/paper/2307.14334/) (2023)
- [Can AI help in screening Viral and COVID-19 pneumonia?](https://mlsanity.com/paper/2003.13145/) (2020)
- [AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model](https://mlsanity.com/paper/2309.16058/) (2023)
- [CXR-CLIP: Toward Large Scale Chest X-ray Language-Image Pre-training](https://mlsanity.com/paper/2310.13292/) (2023)
- [S-CLIP: Semi-supervised Vision-Language Learning using Few Specialist Captions](https://mlsanity.com/paper/2305.14095/) (2023)
- [All in One: Exploring Unified Vision-Language Tracking with Multi-Modal Alignment](https://mlsanity.com/paper/2307.03373/) (2023)

## Built on by (most cited first, 8 of 8)

- [Learning Contrastive Multimodal Fusion with Improved Modality Dropout for Disease Detection and Prediction](https://mlsanity.com/paper/2509.18284/) (2025)
- [EchoingECG: An Electrocardiogram Cross-Modal Model for Echocardiogram Tasks](https://mlsanity.com/paper/2509.25791/) (2025)
- [ChatEXAONEPath: An Expert-level Multimodal Large Language Model for Histopathology Using Whole Slide Images](https://mlsanity.com/paper/2504.13023/) (2025)
- [Multimodal Medical Image Binding via Shared Text Embeddings](https://mlsanity.com/paper/2506.18072/) (2025)
- [X2CT-CLIP: Enable Multi-Abnormality Detection in Computed Tomography from Chest Radiography via Tri-Modal Contrastive Learning](https://mlsanity.com/paper/2503.02162/) (2025)
- [ProbMed: A Probabilistic Framework for Medical Multimodal Binding](https://mlsanity.com/paper/2509.25711/) (2025)
- [Hub-Spectral Activation of Latent Multimodal Knowledge](https://mlsanity.com/paper/2609.17094/) (2026)
- [TRACE: Expert-Aligned ECG Representation Learning with Rigorous Benchmarking and Real-World Validation in Acute Cardiac Care](https://mlsanity.com/paper/2609.34088/) (2026)

## Abstract (arXiv)

Medical vision-language pretraining models (VLPM) have achieved remarkable progress in fusing chest X-rays (CXR) with clinical texts, introducing image-text data binding approaches that enable zero-shot learning and downstream clinical tasks. However, the current landscape lacks the holistic integration of additional medical modalities, such as electrocardiograms (ECG). We present MEDBind (Medical Electronic patient recorD), which learns joint embeddings across CXR, ECG, and medical text. Using text data as the central anchor, MEDBind features tri-modality binding, delivering competitive performance in top-K retrieval, zero-shot, and few-shot benchmarks against established VLPM, and the ability for CXR-to-ECG zero-shot classification and retrieval. This seamless integration is achieved through combination of contrastive loss on modality-text pairs with our proposed contrastive loss function, Edge-Modality Contrastive Loss, fostering a cohesive embedding space for CXR, ECG, and text. Finally, we demonstrate that MEDBind can improve downstream tasks by directly integrating CXR and ECG embeddings into a large-language model for multimodal prompt tuning.