ShapKO: Shapley-Adaptive Modality Knockout for Robust Multimodal Learning
Not on the map. Why?
Abstract (arXiv)
Multimodal medical models often degrade when inputs are missing, a common scenario in real-world clinical workflows. Separately, even when all modalities are present, modality dominance is observed during training, where optimization over-relies on a highly predictive modality and undertrains complementary sources, resulting in poor robustness under partial availability. While training-time modality knockout improves missing-modality robustness, existing approaches use static masking rates that cannot adapt to evolving modality utility during training. We introduce ShapKO (Shapley-Adaptive Modality Knockout), a dynamic training strategy that learns modality-specific knockout probabilities based on validation utility. ShapKO periodically evaluates performance across modality subsets, estimates modality importance via Shapley values, and updates masking probabilities to suppress dominant modalities more frequently. This adaptive process promotes complementary representations, while requiring no architectural modifications. We evaluate ShapKO on three datasets covering multitask clinical classification, survival prediction, and cancer detection. ShapKO consistently improves performance under modality absence and yields interpretable trajectories of learned masking behavior. Code is available at: https://github.com/sumona00/ShapKO
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 28 references that are in the map. First posted on arXiv 10 Jul 2026.
Builds on
18 of 28 references.
References · 18
- 2022Balanced Multimodal Learning via On-the-fly Gradient Modulationcited by 259
- 2018Multimodal Generative Models for Scalable Weakly-Supervised Learningcited by 200
- 2016HeMIS: Hetero-Modal Image Segmentationcited by 102
- 2021Sampling Permutations for Shapley Value Estimationcited by 101
- 2024Deep Multimodal Learning with Missing Modality: A Surveycited by 89
- 2023Multimodal Representation Learning by Alternating Unimodal Adaptationcited by 74
- 2023Boosting Multi-modal Model Performance with Adaptive Gradient Modulationcited by 71
- 2019Hetero-Modal Variational Encoder-Decoder for Joint Modality Completion and Segmentationcited by 68
- 2024Classifier-guided Gradient Modulation for Enhanced Multimodal Learningcited by 34
- 2022Mitigating Modality Collapse in Multimodal VAEs via Impartial Optimizationcited by 32
- 2022Survival Prediction of Brain Cancer with Incomplete Radiology, Pathology, Genomics, and Demographic Datacited by 25
- 2025Regression-adjusted Monte Carlo Estimators for Shapley Values and Probabilistic Valuescited by 19
6 more
- 2017Learning End-to-end Multimodal Sensor Policies for Autonomous Navigationcited by 16
- 2024Improving Multimodal Learning with Multi-Loss Gradient Modulationcited by 16
- 2024FlexCare: Leveraging Cross-Task Synergy for Flexible Multimodal Healthcare Predictioncited by 6
- 2023What Makes for Robust Multi-Modal Models in the Face of Missing Modalities?cited by 5
- 2024Knockout: A simple way to handle missing inputscited by 4
- 2021Investigating the Effect of Sensor Modalities in Multi-Sensor Detection-Prediction Modelscited by 3
Built on by
Not cited yet.