mlsanityData to 14 Oct 2026

Robust Multimodal Learning via Entropy-Gated Contrastive Fusion

arXiv 2505.15417 ↗First posted 21 May 2025stat.ML cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Real-world multimodal systems routinely face missing-input scenarios, and in reality, robots lose audio in a factory or a clinical record omits lab tests at inference time. Standard fusion layers either preserve robustness or calibration but never both. We introduce Adaptive Entropy-Gated Contrastive Fusion (AECF), a single light-weight layer that (i) adapts its entropy coefficient per instance, (ii) enforces monotone calibration across all modality subsets, and (iii) drives a curriculum mask directly from training-time entropy. On AV-MNIST and MS-COCO, AECF improves masked-input mAP by +18 pp at a 50% drop rate while reducing ECE by up to 200%, yet adds 1% run-time. All back-bones remain frozen, making AECF an easy drop-in layer for robust, calibrated multimodal inference.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 8 of its 28 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

8 of 28 references.

References · 8

  1. 2017On Calibration of Modern Neural Networkscited by 4,960
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  6. 2017Efficient Regret Minimization in Non-Convex Gamescited by 28
  7. 2023Calibrating Multimodal Learningcited by 17
  8. 2018A Local Regret in Nonconvex Online Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.