mlsanityData to 7 Oct 2026

Multimodal Data Integration for Precision Oncology: Challenges and Future Directions

arXiv 2406.19611 ↗First posted 28 Jun 2024q-bio.QM cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

The essence of precision oncology lies in its commitment to tailor targeted treatments and care measures to each patient based on the individual characteristics of the tumor. The inherent heterogeneity of tumors necessitates gathering information from diverse data sources to provide valuable insights from various perspectives, fostering a holistic comprehension of the tumor. Over the past decade, multimodal data integration technology for precision oncology has made significant strides, showcasing remarkable progress in understanding the intricate details within heterogeneous data modalities. These strides have exhibited tremendous potential for improving clinical decision-making and model interpretation, contributing to the advancement of cancer care and treatment. Given the rapid progress that has been achieved, we provide a comprehensive overview of about 300 papers detailing cutting-edge multimodal data integration techniques in precision oncology. In addition, we conclude the primary clinical applications that have reaped significant benefits, including early assessment, diagnosis, prognosis, and biomarker discovery. Finally, derived from the findings of this survey, we present an in-depth analysis that explores the pivotal challenges and reveals essential pathways for future research in the field of multimodal data integration for precision oncology.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 111 of its 368 references that are in the map. First posted on arXiv 28 Jun 2024.

Builds on

111 of 368 references.

References · 111

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  5. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
  6. 2018Attention-based Deep Multiple Instance Learningcited by 1,077
  7. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  8. 2021The Medical Segmentation Decathloncited by 620
  9. 2017Self-Normalizing Neural Networkscited by 591
  10. 2022Continual Test-Time Domain Adaptationcited by 520
  11. 2022Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learningcited by 357
  12. 2023Segment Anything Model for Medical Image Analysis: an Experimental Studycited by 291
38 more
  1. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  2. 2021TransBTS: Multimodal Brain Tumor Segmentation Using Transformercited by 265
  3. 2016Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentationcited by 263
  4. 2022Contrastive Test-Time Adaptationcited by 258
  5. 2022Transformers in Medical Imaging: A Surveycited by 245
  6. 2019MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentationcited by 195
  7. 2021nnFormer: Interleaved Transformer for Volumetric Segmentationcited by 179
  8. 2021ResViT: Residual vision transformers for multi-modal medical image synthesiscited by 177
  9. 2019Pathomic Fusion: An Integrated Framework for Fusing Histopathology and Genomic Features for Cancer Diagnosis and Prognosiscited by 165
  10. 2019Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentationcited by 162
  11. 2023Multi-modal Learning with Missing Modality via Shared-Specific Feature Modellingcited by 159
  12. 2018Handling Incomplete Heterogeneous Data using VAEscited by 148
  13. 2015Brain Tumor Segmentation with Deep Neural Networkscited by 137
  14. 2022Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)cited by 115
  15. 2017Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuningcited by 110
  16. 2022LViT: Language meets Vision Transformer in Medical Image Segmentationcited by 110
  17. 2016HeMIS: Hetero-Modal Image Segmentationcited by 102
  18. 2019Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Networkcited by 98
  19. 2019Deep Neural Networks Improve Radiologists' Performance in Breast Cancer Screeningcited by 94
  20. 2022mmFormer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentationcited by 94
  21. 2023Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Predictioncited by 92
  22. 2020Hi-Net: Hybrid-fusion Network for Multi-modal MR Image Synthesiscited by 86
  23. 2023Multimodal Optimal Transport-based Co-Attention Transformer with Global Structure Consistency for Survival Predictioncited by 86
  24. 2023Quantifying & Modeling Multimodal Interactions: An Information Decomposition Frameworkcited by 81
  25. 2022M3Care: Learning with Missing Modalities in Multimodal Healthcare Datacited by 71
  26. 2023Cross-Modal Translation and Alignment for Survival Analysiscited by 70
  27. 2022Integrated multimodal artificial intelligence framework for healthcare applicationscited by 69
  28. 2019Hetero-Modal Variational Encoder-Decoder for Joint Modality Completion and Segmentationcited by 68
  29. 2021Knowledge distillation from multi-modal to mono-modal segmentation networkscited by 68
  30. 2020Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusioncited by 67
  31. 2024The 2024 Brain Tumor Segmentation (BraTS) Challenge: Glioma Segmentation on Post-treatment MRIcited by 65
  32. 2023Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Reviewcited by 62
  33. 2023Large AI Models in Health Informatics: Applications, Challenges, and the Futurecited by 61
  34. 2017MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Networkcited by 56
  35. 2023M3AE: Multimodal Representation Learning for Brain Tumor Segmentation with Missing Modalitiescited by 56
  36. 2021Modality-aware Mutual Learning for Multi-modal Medical Image Segmentationcited by 47
  37. 2023Learnable Cross-modal Knowledge Distillation for Multi-modal Learning with Missing Modalitycited by 44
  38. 2021ACN: Adversarial Co-training Network for Brain Tumor Segmentation with Missing Modalitiescited by 43

50 most cited of 111 All 111 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 1.

10 papers.

  1. 2025DrugPilot: LLM-based Parameterized Reasoning Agent for Drug Discoverycited by 18
  2. 2025Distilled Prompt Learning for Incomplete Multimodal Survival Predictioncited by 10
  3. 2025From Classical Machine Learning to Emerging Foundation Models: Review on Multimodal Data Integration for Cancer Researchcited by 6
  4. 2025No Modality Left Behind: Adapting to Missing Modalities via Knowledge Distillation for Brain Tumor Segmentationcited by 4
  5. 2025A Multimodal Foundation Model to Enhance Generalizability and Data Efficiency for Pan-cancer Prognosis Predictioncited by 2
  6. 2025Multimodal Disease Progression Modeling via Spatiotemporal Disentanglement and Multiscale Alignmentcited by 1
  7. 2025REACT-KD: Region-Aware Cross-modal Topological Knowledge Distillation for Interpretable Medical Image Classificationcited by 0
  8. 2025Systematic Evaluation of Preprocessing Techniques for Accurate Image Registration in Digital Pathologycited by 0
  9. 2025Deep Learning-Based Computer Vision Models for Early Cancer Detection Using Multimodal Medical Imaging and Radiogenomic Integration Frameworkscited by 0
  10. 2026CIGTSurv: Clinical Information Guided Tri-modal Survival Prediction with Local Prototype Association and Global Feature Alignmentcited by 0