# MMIST-ccRCC: A Real World Medical Dataset for the Development of Multi-Modal Systems

arXiv: https://arxiv.org/abs/2405.01658 · HTML: https://mlsanity.com/paper/2405.01658/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 27 references that are in the map.
- First posted on arXiv 2 May 2024.

## Builds on (most cited first, 6 of 6)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [Med3D: Transfer Learning for 3D Medical Image Analysis](https://mlsanity.com/paper/1904.00625/) (2019)
- [ContIG: Self-supervised Multimodal Contrastive Learning for Medical Imaging with Genetics](https://mlsanity.com/paper/2111.13424/) (2021)
- [Survival Prediction of Brain Cancer with Incomplete Radiology, Pathology, Genomics, and Demographic Data](https://mlsanity.com/paper/2203.04419/) (2022)
- [Cascaded Multi-Modal Mixing Transformers for Alzheimer's Disease Classification with Incomplete Data](https://mlsanity.com/paper/2210.00255/) (2022)

## Built on by (most cited first, 5 of 5)

- [MozzaVID: Mozzarella Volumetric Image Dataset](https://mlsanity.com/paper/2412.04880/) (2024)
- [MIL vs. Aggregation: Evaluating Patient-Level Survival Prediction Strategies Using Graph-Based Learning](https://mlsanity.com/paper/2503.23042/) (2025)
- [Bayesian Integration of Nonlinear Incomplete Clinical Data](https://mlsanity.com/paper/2602.01924/) (2026)
- [MedMIX: Modality-Internal Expert Fusion for Multimodal Medical Diagnosis](https://mlsanity.com/paper/2605.16639/) (2026)
- [Design Choices for Multimodal Transformer Learning with Missing and Longitudinal Healthcare Data](https://mlsanity.com/paper/2610.14436/) (2026)

## Abstract (arXiv)

The acquisition of different data modalities can enhance our knowledge and understanding of various diseases, paving the way for a more personalized healthcare. Thus, medicine is progressively moving towards the generation of massive amounts of multi-modal data (e.g, molecular, radiology, and histopathology). While this may seem like an ideal environment to capitalize data-centric machine learning approaches, most methods still focus on exploring a single or a pair of modalities due to a variety of reasons: i) lack of ready to use curated datasets; ii) difficulty in identifying the best multi-modal fusion strategy; and iii) missing modalities across patients. In this paper we introduce a real world multi-modal dataset called MMIST-CCRCC that comprises 2 radiology modalities (CT and MRI), histopathology, genomics, and clinical data from 618 patients with clear cell renal cell carcinoma (ccRCC). We provide single and multi-modal (early and late fusion) benchmarks in the task of 12-month survival prediction in the challenging scenario of one or more missing modalities for each patient, with missing rates that range from 26% for genomics data to more than 90% for MRI. We show that even with such severe missing rates the fusion of modalities leads to improvements in the survival forecasting. Additionally, incorporating a strategy to generate the latent representations of the missing modalities given the available ones further improves the performance, highlighting a potential complementarity across modalities. Our dataset and code are available here: https://multi-modal-ist.github.io/datasets/ccRCC