mlsanityData to 14 Oct 2026

UG-UMRE: Uncertainty-Guided Modality Augmentation and Distributional Calibration for Unified Multimodal Relation Extraction

arXiv 2608.04949 ↗First posted 5 Aug 2026cs.CV cs.CL cs.ITcited by 0

Not on the map. Why?

Abstract (arXiv)

Unified Multimodal Relation Extraction (UMRE) aims to identify intra-modal and cross-modal relations between textual entities and visual objects. However, existing UMRE studies still encounter two critical issues: ignoring inherent aleatoric uncertainty causes noise propagation, and deep-seated heterogeneity between distinct modal distributions hinders alignment. To address these issues, we propose the Uncertainty-Guided UMRE Network (UG-UMRE). Specifically, we design an Uncertainty-Driven Unimodal Augmentation (UDUA) module, which models features as Gaussian distributions based on the Variational Information Bottleneck. By incorporating an uncertainty-aware self-supervised contrastive learning mechanism, UDUA effectively filters out noise while maintaining semantic consistency. Furthermore, we introduce the Joint Aleatoric Uncertainty Alignment (JAUA) module as a global semantic pre-calibration mechanism. JAUA leverages probabilistic distribution consistency to construct a shared latent space, eliminating the distributional gap by synchronizing cross-modal statistical properties, thereby laying a robust foundation for fine-grained interaction. Experiments on three benchmark datasets (UMRE, MORE, and MNRE) demonstrate that UG-UMRE achieves state-of-the-art performance. Further analysis validates the pluggable and effective performance of the proposed UDUA and JAUA modules.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 50 references that are in the map. First posted on arXiv 5 Aug 2026.

Builds on

15 of 50 references.

References · 15

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  5. 2016Deep Variational Information Bottleneckcited by 930
  6. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  7. 2022MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Modelcited by 41
  8. 2023Scalable Optimal Transport Methods in Machine Learning: A Contemporary Surveycited by 27
  9. 2023Exploiting Multi-Label Correlation in Label Distribution Learningcited by 20
  10. 2023Information Screening whilst Exploiting! Multimodal Relation Extraction with Feature Denoising and Multimodal Topic Modelingcited by 16
  11. 2025Collaborative Multi-LoRA Experts with Achievement-based Multi-Tasks Loss for Unified Multimodal Information Extractioncited by 7
  12. 2023MORE: A Multimodal Object-Entity Relation Extraction Dataset with a Benchmark Evaluationcited by 5
3 more
  1. 2025A Survey on Algorithmic Developments in Optimal Transport Problem with Applicationscited by 4
  2. 2025REMOTE: A Unified Multimodal Relation Extraction Framework with Multilevel Optimal Transport and Mixture-of-Expertscited by 2
  3. 2026Trustworthy Federated Label Distribution Learning under Annotation Quality Disparitycited by 1

Built on by

Not cited yet.