# MoSE: Modality Split and Ensemble for Multimodal Knowledge Graph Completion

arXiv: https://arxiv.org/abs/2210.08821 · HTML: https://mlsanity.com/paper/2210.08821/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 17 Oct 2022.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge](https://mlsanity.com/paper/1906.00067/) (2019)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [KG-BERT: BERT for Knowledge Graph Completion](https://mlsanity.com/paper/1909.03193/) (2019)
- [Canonical Tensor Decomposition for Knowledge Base Completion](https://mlsanity.com/paper/1806.07297/) (2018)
- [Multi-Modal Knowledge Graph Construction and Application: A Survey](https://mlsanity.com/paper/2202.05786/) (2022)
- [Image-embodied Knowledge Representation Learning](https://mlsanity.com/paper/1609.07028/) (2016)
- [Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completion](https://mlsanity.com/paper/2205.02357/) (2022)
- [Multilingual Knowledge Graph Completion via Ensemble Knowledge Transfer](https://mlsanity.com/paper/2010.03158/) (2020)

## Built on by (most cited first, 14 of 14)

- [A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodal](https://mlsanity.com/paper/2212.05767/) (2022)
- [Mosaic of Modalities: A Comprehensive Benchmark for Multimodal Graph Learning](https://mlsanity.com/paper/2406.16321/) (2024)
- [Learning on Multimodal Graphs: A Survey](https://mlsanity.com/paper/2402.05322/) (2024)
- [Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation Learning](https://mlsanity.com/paper/2405.16869/) (2024)
- [Unleashing the Power of Imbalanced Modality Information for Multi-modal Knowledge Graph Completion](https://mlsanity.com/paper/2402.15444/) (2024)
- [Structure Guided Multi-modal Pre-trained Transformer for Knowledge Graph Reasoning](https://mlsanity.com/paper/2307.03591/) (2023)
- [Scalable Incomplete Multi-View Clustering with Structure Alignment](https://mlsanity.com/paper/2308.16541/) (2023)
- [Zero-Shot Relational Learning for Multimodal Knowledge Graphs](https://mlsanity.com/paper/2404.06220/) (2024)
- [Can Multimodal Large Language Model Think Analogically?](https://mlsanity.com/paper/2411.01307/) (2024)
- [VL-KGE: Vision-Language Models Meet Knowledge Graph Embeddings](https://mlsanity.com/paper/2603.02435/) (2026)
- [MACO: A Modality Adversarial and Contrastive Framework for Modality-missing Multi-modal Knowledge Graph Completion](https://mlsanity.com/paper/2308.06696/) (2023)
- [Hyper-modal Imputation Diffusion Embedding with Dual-Distillation for Federated Multimodal Knowledge Graph Completion](https://mlsanity.com/paper/2506.22036/) (2025)
- [CMKL: Modality-Aware Continual Learning for Evolving Biomedical Knowledge Graphs](https://mlsanity.com/paper/2605.10510/) (2026)
- [PrimeKG-CL: A Continual Graph Learning Benchmark on Evolving Biomedical Knowledge Graphs](https://mlsanity.com/paper/2605.10529/) (2026)

## Abstract (arXiv)

Multimodal knowledge graph completion (MKGC) aims to predict missing entities in MKGs. Previous works usually share relation representation across modalities. This results in mutual interference between modalities during training, since for a pair of entities, the relation from one modality probably contradicts that from another modality. Furthermore, making a unified prediction based on the shared relation representation treats the input in different modalities equally, while their importance to the MKGC task should be different. In this paper, we propose MoSE, a Modality Split representation learning and Ensemble inference framework for MKGC. Specifically, in the training phase, we learn modality-split relation embeddings for each modality instead of a single modality-shared one, which alleviates the modality interference. Based on these embeddings, in the inference phase, we first make modality-split predictions and then exploit various ensemble methods to combine the predictions with different weights, which models the modality importance dynamically. Experimental results on three KG datasets show that MoSE outperforms state-of-the-art MKGC methods. Codes are available at https://github.com/OreOZhao/MoSE4MKGC.