# Multimodal Doctor-in-the-Loop: A Clinically-Guided Explainable Framework for Predicting Pathological Response in Non-Small Cell Lung Cancer

arXiv: https://arxiv.org/abs/2505.01390 · HTML: https://mlsanity.com/paper/2505.01390/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 33 references that are in the map.
- First posted on arXiv 2 May 2025.

## Builds on (most cited first, 10 of 10)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [A Systematic Review of Intermediate Fusion in Multimodal Deep Learning for Biomedical Applications](https://mlsanity.com/paper/2408.02686/) (2024)
- [Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlooks](https://mlsanity.com/paper/2410.02331/) (2024)
- [Multimodal Explainability via Latent Shift applied to COVID-19 stratification](https://mlsanity.com/paper/2212.14084/) (2022)
- [Multi-objective optimization determines when, which and how to fuse deep networks: an application to predict COVID-19 outcomes](https://mlsanity.com/paper/2204.03772/) (2022)
- [Multi-Dataset Multi-Task Learning for COVID-19 Prognosis](https://mlsanity.com/paper/2405.13771/) (2024)
- [A graph neural network-based model with Out-of-Distribution Robustness for enhancing Antiretroviral Therapy Outcome Prediction for HIV-1](https://mlsanity.com/paper/2312.17506/) (2023)
- [Doctor-in-the-Loop: An Explainable, Multi-View Deep Learning Framework for Predicting Pathological Response in Non-Small Cell Lung Cancer](https://mlsanity.com/paper/2502.17503/) (2025)
- [Class Balancing Diversity Multimodal Ensemble for Alzheimer's Disease Diagnosis and Early Detection](https://mlsanity.com/paper/2410.10374/) (2024)

## Built on by (most cited first, 2 of 2)

- [Longitudinal NSCLC Treatment Progression via Multimodal Generative Models](https://mlsanity.com/paper/2603.06147/) (2026)
- [Learning from Limited and Incomplete Data: A Multimodal Framework for Predicting Pathological Response in NSCLC](https://mlsanity.com/paper/2603.15100/) (2026)

## Abstract (arXiv)

This study proposes a novel approach combining Multimodal Deep Learning with intrinsic eXplainable Artificial Intelligence techniques to predict pathological response in non-small cell lung cancer patients undergoing neoadjuvant therapy. Due to the limitations of existing radiomics and unimodal deep learning approaches, we introduce an intermediate fusion strategy that integrates imaging and clinical data, enabling efficient interaction between data modalities. The proposed Multimodal Doctor-in-the-Loop method further enhances clinical relevance by embedding clinicians' domain knowledge directly into the training process, guiding the model's focus gradually from broader lung regions to specific lesions. Results demonstrate improved predictive accuracy and explainability, providing insights into optimal data integration strategies for clinical applications.