Backbone-Conditional Behavior of Modality Gating in Multi-Modal Prostate MRI Segmentation: A 5-Fold Cross-Validation and Gate Mechanism Analysis
On the map: State-space sequence models (S4, Mamba) → Mamba for medical image analysis
Abstract (arXiv)
Robust segmentation of clinically significant prostate cancer (csPCa) on multi-parametric MRI must tolerate frequent degradation of its most informative diffusion sequences. Multi-modal fusion commonly employs learned modality gating under the assumption that gates implement per-sample modality quality routing -- rarely tested directly. We ask how gating behaves across backbone architectures. We systematically analyze modality-isolated gated fusion (MIGF) for csPCa segmentation on two backbones (nnU-Net and Mamba) using PI-CAI (n=1500), with cross-cohort validation on Prostate158 (n=158): a factorial ablation over gating, modality dropout, and deep supervision under 5-fold cross-validation (180 trained models), plus a gate-weight and counterfactual analysis of 30 trained gating models. Modality gating is backbone-conditional. On nnU-Net, adding gating reduces the ranking score (marginal effect -0.037; gating configurations p<0.05), whereas on Mamba the gating-plus-dropout configuration improves it (+0.024, p=0.037). Gate-weight analysis explains this: nnU-Net gates collapse into a near-static modality prior (across-case SD 0.0033), while Mamba gates retain sample-dependent variation (0.0365, ~11x larger, non-overlapping); replacing per-sample gates with their training-set mean leaves nnU-Net unchanged but degrades Mamba. Modality dropout is the only component beneficial on both backbones. Under cross-cohort shift, convolutional backbones collapse to case-level specificity near zero, whereas Mamba retains it (MIGF-Mamba highest, 0.31). Learned modality gates do not universally perform per-sample quality routing; their effective behavior is conditional on the backbone's inherent modality awareness. Among tested configurations, MIGF-Mamba is the most cross-cohort robust, and training-time modality dropout is the only component beneficial across both backbones.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Mamba for medical image analysis, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 3 of its 29 references that are in the map. First posted on arXiv 12 Apr 2026.
Builds on
3 of 29 references.
References · 3
- 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
- 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
- 2022mmFormer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentationcited by 94
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Modality Translation and Registration of MR and Ultrasound Images Using Diffusion Models
- 2023Modality Cycles with Masked Conditional Diffusion for Unsupervised Anomaly Segmentation in MRI
- 2026Align then Refine: Text-Guided 3D Prostate Lesion Segmentation
- 2024Generalizable Single-Source Cross-modality Medical Image Segmentation via Invariant Causal Mechanisms
- 2023Cancer-Net PCa-Gen: Synthesis of Realistic Prostate Diffusion Weighted Imaging Data via Anatomic-Conditional Controlled Latent Diffusion
- 2026Maximizing T2-Only Prostate Cancer Localization from Expected Diffusion Weighted Imaging
- 2025AMM-Diff: Adaptive Multi-Modality Diffusion Network for Missing Modality Imputation
- 2024CriDiff: Criss-cross Injection Diffusion Framework via Generative Pre-train for Prostate Segmentation