# Vision, Deduction and Alignment: An Empirical Study on Multi-modal Knowledge Graph Alignment

arXiv: https://arxiv.org/abs/2302.08774 · HTML: https://mlsanity.com/paper/2302.08774/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 33 references that are in the map.
- First posted on arXiv 17 Feb 2023.

## Builds on (most cited first, 20 of 20)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [A Benchmarking Study of Embedding-based Entity Alignment for Knowledge Graphs](https://mlsanity.com/paper/2003.07743/) (2020)
- [Relation-Aware Entity Alignment for Heterogeneous Knowledge Graphs](https://mlsanity.com/paper/1908.08210/) (2019)
- [Exploring and Evaluating Attributes, Values, and Structures for Entity Alignment](https://mlsanity.com/paper/2010.03249/) (2020)
- [Are we ready for a new paradigm shift? A Survey on Visual Deep MLP](https://mlsanity.com/paper/2111.04060/) (2021)
- [Visual Pivoting for (Unsupervised) Entity Alignment](https://mlsanity.com/paper/2009.13603/) (2020)
- [Aligning Cross-Lingual Entities with Multi-Aspect Information](https://mlsanity.com/paper/1910.06575/) (2019)
- [Jointly Learning Entity and Relation Representations for Entity Alignment](https://mlsanity.com/paper/1909.09317/) (2019)
- [Contrastive Learning with Hard Negative Entities for Entity Set Expansion](https://mlsanity.com/paper/2204.07789/) (2022)
- [AliCoCo: Alibaba E-commerce Cognitive Concept Net](https://mlsanity.com/paper/2003.13230/) (2020)
- [Automatic Context Pattern Generation for Entity Set Expansion](https://mlsanity.com/paper/2207.08087/) (2022)
- [Knowing the No-match: Entity Alignment with Dangling Cases](https://mlsanity.com/paper/2106.02248/) (2021)
- [An Industry Evaluation of Embedding-based Entity Alignment](https://mlsanity.com/paper/2010.11522/) (2020)
- [Embracing Ambiguity: Improving Similarity-oriented Tasks with Contextual Synonym Knowledge](https://mlsanity.com/paper/2211.10997/) (2022)
- [Active Relation Discovery: Towards General and Label-aware Open Relation Extraction](https://mlsanity.com/paper/2211.04215/) (2022)
- [Learning Purified Feature Representations from Task-irrelevant Labels](https://mlsanity.com/paper/2102.10955/) (2021)

## Built on by (most cited first, 19 of 19)

- [Towards Real-World Writing Assistance: A Chinese Character Checking Benchmark with Faked and Misspelled Characters](https://mlsanity.com/paper/2311.11268/) (2023)
- [Rethinking Uncertainly Missing and Ambiguous Visual Modality in Multi-Modal Entity Alignment](https://mlsanity.com/paper/2307.16210/) (2023)
- [ReLiK: Retrieve and LinK, Fast and Accurate Entity Linking and Relation Extraction on an Academic Budget](https://mlsanity.com/paper/2408.00103/) (2024)
- [MESED: A Multi-modal Entity Set Expansion Dataset with Fine-grained Semantic Classes and Hard Negative Entities](https://mlsanity.com/paper/2307.14878/) (2023)
- [Bidirectional End-to-End Learning of Retriever-Reader Paradigm for Entity Linking](https://mlsanity.com/paper/2306.12245/) (2023)
- [Multi-Modal Knowledge Graph Transformer Framework for Multi-Modal Entity Alignment](https://mlsanity.com/paper/2310.06365/) (2023)
- [Noise-powered Multi-modal Knowledge Graph Representation Framework](https://mlsanity.com/paper/2403.06832/) (2024)
- [ASGEA: Exploiting Logic Rules from Align-Subgraphs for Entity Alignment](https://mlsanity.com/paper/2402.11000/) (2024)
- [CSPO: Cross-Market Synergistic Stock Price Movement Forecasting with Pseudo-volatility Optimization](https://mlsanity.com/paper/2503.22740/) (2025)
- [From Token to Line: Enhancing Code Generation with a Long-Term Perspective](https://mlsanity.com/paper/2504.07433/) (2025)
- [TangramPuzzle: Evaluating Multimodal Large Language Models with Compositional Spatial Reasoning](https://mlsanity.com/paper/2601.16520/) (2026)
- [From Retrieval to Generation: Efficient and Effective Entity Set Expansion](https://mlsanity.com/paper/2304.03531/) (2023)
- [Lambda: Learning Matchable Prior For Entity Alignment with Unlabeled Dangling Cases](https://mlsanity.com/paper/2403.10978/) (2024)
- [MDIT: A Model-free Data Interpolation Method for Diverse Instruction Tuning](https://mlsanity.com/paper/2504.07288/) (2025)
- [Universal Multi-modal Entity Alignment via Iteratively Fusing Modality Similarity Paths](https://mlsanity.com/paper/2310.05364/) (2023)
- [Teaching According to Talents! Instruction Tuning LLMs with Competence-Aware Curriculum Learning](https://mlsanity.com/paper/2509.13790/) (2025)
- [Investigating Graph Structure Information for Entity Alignment with Dangling Cases](https://mlsanity.com/paper/2304.04718/) (2023)
- [Corrections Meet Explanations: A Unified Framework for Explainable Grammatical Error Correction](https://mlsanity.com/paper/2502.15261/) (2025)
- [Knowing When to Trust Images: Reliability-Aware Multi-modal Entity Alignment](https://mlsanity.com/paper/2609.23267/) (2026)

## Abstract (arXiv)

Entity alignment (EA) for knowledge graphs (KGs) plays a critical role in knowledge engineering. Existing EA methods mostly focus on utilizing the graph structures and entity attributes (including literals), but ignore images that are common in modern multi-modal KGs. In this study we first constructed Multi-OpenEA -- eight large-scale, image-equipped EA benchmarks, and then evaluated some existing embedding-based methods for utilizing images. In view of the complementary nature of visual modal information and logical deduction, we further developed a new multi-modal EA method named LODEME using logical deduction and multi-modal KG embedding, with state-of-the-art performance achieved on Multi-OpenEA and other existing multi-modal EA benchmarks.