mlsanityData to 14 Oct 2026

Collaboration of Fusion and Independence: Hypercomplex-driven Robust Multi-Modal Knowledge Graph Completion

arXiv 2509.23714 ↗First posted 28 Sep 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Multi-modal knowledge graph completion (MMKGC) aims to discover missing facts in multi-modal knowledge graphs (MMKGs) by leveraging both structural relationships and diverse modality information of entities. Existing MMKGC methods follow two multi-modal paradigms: fusion-based and ensemble-based. Fusion-based methods employ fixed fusion strategies, which inevitably leads to the loss of modality-specific information and a lack of flexibility to adapt to varying modality relevance across contexts. In contrast, ensemble-based methods retain modality independence through dedicated sub-models but struggle to capture the nuanced, context-dependent semantic interplay between modalities. To overcome these dual limitations, we propose a novel MMKGC method M-Hyper, which achieves the coexistence and collaboration of fused and independent modality representations. Our method integrates the strengths of both paradigms, enabling effective cross-modal interactions while maintaining modality-specific information. Inspired by ``quaternion'' algebra, we utilize its four orthogonal bases to represent multiple independent modalities and employ the Hamilton product to efficiently model pair-wise interactions among them. Specifically, we introduce a Fine-grained Entity Representation Factorization (FERF) module and a Robust Relation-aware Modality Fusion (R2MF) module to obtain robust representations for three independent modalities and one fused modality. The resulting four modality representations are then mapped to the four orthogonal bases of a biquaternion (a hypercomplex extension of quaternion) for comprehensive modality interaction. Extensive experiments indicate its state-of-the-art performance, robustness, and computational efficiency.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 41 references that are in the map. First posted on arXiv 28 Sep 2025.

Builds on

21 of 41 references.

References · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2016Complex Embeddings for Simple Link Predictioncited by 902
  4. 2019RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Spacecited by 898
  5. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  6. 2019KGAT: Knowledge Graph Attention Network for Recommendationcited by 343
  7. 2019Quaternion Knowledge Graph Embeddingscited by 175
  8. 2018Canonical Tensor Decomposition for Knowledge Base Completioncited by 150
  9. 2022A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodalcited by 106
  10. 2019MMKG: Multi-Modal Knowledge Graphscited by 105
  11. 2016Image-embodied Knowledge Representation Learningcited by 68
  12. 2024Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Surveycited by 53
9 more
  1. 2023IMF: Interactive Multimodal Fusion Model for Link Predictioncited by 28
  2. 2023Rethinking Uncertainly Missing and Ambiguous Visual Modality in Multi-Modal Entity Alignmentcited by 21
  3. 2021BiQUE: Biquaternionic Embeddings of Knowledge Graphscited by 19
  4. 2022Knowledge Graph Completion with Pre-trained Multimodal Transformer and Twins Negative Samplingcited by 18
  5. 2022Tele-Knowledge Pre-training for Fault Analysiscited by 18
  6. 2024Tokenization, Fusion, and Augmentation: Towards Fine-grained Multi-modal Entity Representationcited by 14
  7. 2024Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation Learningcited by 13
  8. 2023Learning Representations of Bi-level Knowledge Graphs for Reasoning beyond Link Predictioncited by 9
  9. 2024UniHR: Hierarchical Representation Learning for Unified Knowledge Graph Link Predictioncited by 3

Built on by

Not cited yet.