# MUDI: A Multimodal Biomedical Dataset for Understanding Pharmacodynamic Drug-Drug Interactions

arXiv: https://arxiv.org/abs/2506.01478 · HTML: https://mlsanity.com/paper/2506.01478/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 34 references that are in the map.
- First posted on arXiv 2 Jun 2025.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development](https://mlsanity.com/paper/2102.09548/) (2021)
- [Drug-Drug Interaction Extraction from Biomedical Text Using Long Short Term Memory Network](https://mlsanity.com/paper/1701.08303/) (2017)
- [HODDI: A Dataset of High-Order Drug-Drug Interactions for Computational Pharmacovigilance](https://mlsanity.com/paper/2502.06274/) (2025)
- [Provenance-Centered Dataset of Drug-Drug Interactions](https://mlsanity.com/paper/1507.05408/) (2015)
- [BERTChem-DDI : Improved Drug-Drug Interaction Prediction from text using Chemical Structure Information](https://mlsanity.com/paper/2012.11599/) (2020)

## Abstract (arXiv)

Understanding the interaction between different drugs (drug-drug interaction or DDI) is critical for ensuring patient safety and optimizing therapeutic outcomes. Existing DDI datasets primarily focus on textual information, overlooking multimodal data that reflect complex drug mechanisms. In this paper, we (1) introduce MUDI, a large-scale Multimodal biomedical dataset for Understanding pharmacodynamic Drug-drug Interactions, and (2) benchmark learning methods to study it. In brief, MUDI provides a comprehensive multimodal representation of drugs by combining pharmacological text, chemical formulas, molecular structure graphs, and images across 310,532 annotated drug pairs labeled as Synergism, Antagonism, or New Effect. Crucially, to effectively evaluate machine-learning based generalization, MUDI consists of unseen drug pairs in the test set. We evaluate benchmark models using both late fusion voting and intermediate fusion strategies. All data, annotations, evaluation scripts, and baselines are released under an open research license.