Robust Multimodal Learning via Entropy-Gated Contrastive Fusion
Not on the map. Why?
Abstract (arXiv)
Real-world multimodal systems routinely face missing-input scenarios, and in reality, robots lose audio in a factory or a clinical record omits lab tests at inference time. Standard fusion layers either preserve robustness or calibration but never both. We introduce Adaptive Entropy-Gated Contrastive Fusion (AECF), a single light-weight layer that (i) adapts its entropy coefficient per instance, (ii) enforces monotone calibration across all modality subsets, and (iii) drives a curriculum mask directly from training-time entropy. On AV-MNIST and MS-COCO, AECF improves masked-input mAP by +18 pp at a 50% drop rate while reducing ECE by up to 200%, yet adds 1% run-time. All back-bones remain frozen, making AECF an easy drop-in layer for robust, calibrated multimodal inference.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 8 of its 28 references that are in the map. First posted on arXiv 21 May 2025.
Builds on
8 of 28 references.
References · 8
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
- 2017Efficient Regret Minimization in Non-Convex Gamescited by 28
- 2023Calibrating Multimodal Learningcited by 17
- 2018A Local Regret in Nonconvex Online Learningcited by 3
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.