# Rethinking Uncertainly Missing and Ambiguous Visual Modality in Multi-Modal Entity Alignment

arXiv: https://arxiv.org/abs/2307.16210 · HTML: https://mlsanity.com/paper/2307.16210/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 63 references that are in the map.
- First posted on arXiv 30 Jul 2023.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality](https://mlsanity.com/paper/2304.14178/) (2023)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Cross-lingual Entity Alignment via Joint Attribute-Preserving Embedding](https://mlsanity.com/paper/1708.05045/) (2017)
- [A Benchmarking Study of Embedding-based Entity Alignment for Knowledge Graphs](https://mlsanity.com/paper/2003.07743/) (2020)
- [Relation-Aware Entity Alignment for Heterogeneous Knowledge Graphs](https://mlsanity.com/paper/1908.08210/) (2019)
- [MMKG: Multi-Modal Knowledge Graphs](https://mlsanity.com/paper/1903.05485/) (2019)
- [Knowledge Graph Alignment Network with Gated Multi-hop Neighborhood Aggregation](https://mlsanity.com/paper/1911.08936/) (2019)
- [Multi-Channel Graph Neural Network for Entity Alignment](https://mlsanity.com/paper/1908.09898/) (2019)
- [Multi-view Knowledge Graph Embedding for Entity Alignment](https://mlsanity.com/paper/1906.02390/) (2019)
- [Exploring and Evaluating Attributes, Values, and Structures for Entity Alignment](https://mlsanity.com/paper/2010.03249/) (2020)
- [Molecular Contrastive Learning with Chemical Element Knowledge Graph](https://mlsanity.com/paper/2112.00544/) (2021)
- [Visual Pivoting for (Unsupervised) Entity Alignment](https://mlsanity.com/paper/2009.13603/) (2020)
- [Aligning Cross-Lingual Entities with Multi-Aspect Information](https://mlsanity.com/paper/1910.06575/) (2019)
- [OntoZSL: Ontology-enhanced Zero-shot Learning](https://mlsanity.com/paper/2102.07339/) (2021)
- [Neighborhood Matching Network for Entity Alignment](https://mlsanity.com/paper/2005.05607/) (2020)
- [TransEdge: Translating Relation-contextualized Embeddings for Knowledge Graphs](https://mlsanity.com/paper/2004.13579/) (2020)
- [Knowledge-aware Zero-Shot Learning: Survey and Perspective](https://mlsanity.com/paper/2103.00070/) (2021)
- [Multi-modal Contrastive Representation Learning for Entity Alignment](https://mlsanity.com/paper/2209.00891/) (2022)
- [Zero-shot Visual Question Answering using Knowledge Graph](https://mlsanity.com/paper/2107.05348/) (2021)
- [DUET: Cross-modal Semantic Grounding for Contrastive Zero-shot Learning](https://mlsanity.com/paper/2207.01328/) (2022)
- [MEAformer: Multi-modal Entity Alignment Transformer for Meta Modality Hybrid](https://mlsanity.com/paper/2212.14454/) (2022)
- [Zero-shot and Few-shot Learning with Knowledge Graphs: A Comprehensive Survey](https://mlsanity.com/paper/2112.10006/) (2021)
- [ClusterEA: Scalable Entity Alignment with Stochastic Training and Normalized Mini-batch Similarities](https://mlsanity.com/paper/2205.10312/) (2022)
- [Vision, Deduction and Alignment: An Empirical Study on Multi-modal Knowledge Graph Alignment](https://mlsanity.com/paper/2302.08774/) (2023)

## Built on by (most cited first, 21 of 21)

- [Deep Multimodal Learning with Missing Modality: A Survey](https://mlsanity.com/paper/2409.07825/) (2024)
- [Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answering](https://mlsanity.com/paper/2311.06503/) (2023)
- [NativE: Multi-modal Knowledge Graph Completion in the Wild](https://mlsanity.com/paper/2406.17605/) (2024)
- [Towards Real-World Writing Assistance: A Chinese Character Checking Benchmark with Faked and Misspelled Characters](https://mlsanity.com/paper/2311.11268/) (2023)
- [Noise-powered Multi-modal Knowledge Graph Representation Framework](https://mlsanity.com/paper/2403.06832/) (2024)
- [ASGEA: Exploiting Logic Rules from Align-Subgraphs for Entity Alignment](https://mlsanity.com/paper/2402.11000/) (2024)
- [Progressively Modality Freezing for Multi-Modal Entity Alignment](https://mlsanity.com/paper/2407.16168/) (2024)
- [IBMEA: Exploring Variational Information Bottleneck for Multi-modal Entity Alignment](https://mlsanity.com/paper/2407.19302/) (2024)
- [C-MAG: Cascade Multimodal Attributed Graphs for Supply Chain Link Prediction](https://mlsanity.com/paper/2508.08071/) (2025)
- [Gradient Flow of Energy: A General and Efficient Approach for Entity Alignment Decoding](https://mlsanity.com/paper/2401.12798/) (2024)
- [LoginMEA: Local-to-Global Interaction Network for Multi-modal Entity Alignment](https://mlsanity.com/paper/2407.19625/) (2024)
- [Multi-level Matching Network for Multimodal Entity Linking](https://mlsanity.com/paper/2412.10440/) (2024)
- [Learning with Dual-level Noisy Correspondence for Multi-modal Entity Alignment](https://mlsanity.com/paper/2510.18240/) (2025)
- [How Do Optical Flow and Textual Prompts Collaborate to Assist in Audio-Visual Semantic Segmentation?](https://mlsanity.com/paper/2601.08133/) (2026)
- [MACO: A Modality Adversarial and Contrastive Framework for Modality-missing Multi-modal Knowledge Graph Completion](https://mlsanity.com/paper/2308.06696/) (2023)
- [Mitigating Modality Bias in Multi-modal Entity Alignment from a Causal Perspective](https://mlsanity.com/paper/2504.19458/) (2025)
- [Debate with Images: Detecting Deceptive Behaviors in Multimodal Large Language Models](https://mlsanity.com/paper/2512.00349/) (2025)
- [Collaboration of Fusion and Independence: Hypercomplex-driven Robust Multi-Modal Knowledge Graph Completion](https://mlsanity.com/paper/2509.23714/) (2025)
- [FLORA: Unsupervised Knowledge Graph Alignment by Fuzzy Logic](https://mlsanity.com/paper/2510.20467/) (2025)
- [Knowing When to Trust Images: Reliability-Aware Multi-modal Entity Alignment](https://mlsanity.com/paper/2609.23267/) (2026)
- [Silence and Noise: Resolving Suppression and Overvaluation in Multimodal Entity Alignment](https://mlsanity.com/paper/2610.16106/) (2026)

## Abstract (arXiv)

As a crucial extension of entity alignment (EA), multi-modal entity alignment (MMEA) aims to identify identical entities across disparate knowledge graphs (KGs) by exploiting associated visual information. However, existing MMEA approaches primarily concentrate on the fusion paradigm of multi-modal entity features, while neglecting the challenges presented by the pervasive phenomenon of missing and intrinsic ambiguity of visual images. In this paper, we present a further analysis of visual modality incompleteness, benchmarking latest MMEA models on our proposed dataset MMEA-UMVM, where the types of alignment KGs covering bilingual and monolingual, with standard (non-iterative) and iterative training paradigms to evaluate the model performance. Our research indicates that, in the face of modality incompleteness, models succumb to overfitting the modality noise, and exhibit performance oscillations or declines at high rates of missing modality. This proves that the inclusion of additional multi-modal data can sometimes adversely affect EA. To address these challenges, we introduce UMAEA , a robust multi-modal entity alignment approach designed to tackle uncertainly missing and ambiguous visual modalities. It consistently achieves SOTA performance across all 97 benchmark splits, significantly surpassing existing baselines with limited parameters and time consumption, while effectively alleviating the identified limitations of other models. Our code and benchmark data are available at https://github.com/zjukg/UMAEA.