MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts for Multimodal Knowledge Graph Completion
On the map: Diffusion models → Image generation, text-to-image and editing
also in 1 more
Abstract (arXiv)
Multimodal Knowledge Graph Completion (MKGC) requires inferring missing entities from structural, textual, and visual cues. Existing diffusion-based MKGC methods usually denoise directly on raw multimodal features. Such a design forces the denoiser to simultaneously perform relation-dependent cue selection, cross-modal semantic alignment, and structure-aware entity generation, which introduces noisy and semantically inconsistent conditions for diffusion and consequently leads to suboptimal completion performance. To address this limitation, we propose MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts (MGDT), a novel MKGC framework built on an align-then-diffuse paradigm. MGDT first employs a Relation-Adaptive Semantic Routing Mixture-of-Experts (RASR-MoE) module to select relation-relevant multimodal semantic transformation paths and suppress irrelevant modality interference. MGDT then uses a frozen Multimodal Large Language Model (MLLM) as a semantic anchor to align the routed multimodal representations into a unified latent space and reduce cross-modal semantic heterogeneity. Finally, a Knowledge Graph Diffusion Transformer (KGDT) performs graph-conditioned denoising generation in the aligned space to produce the missing entity representation. Experiments on three benchmark datasets show that MGDT consistently outperforms strong baselines.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 15 of its 28 references that are in the map. First posted on arXiv 17 Jul 2026.
Builds on
15 of 28 references.
Closest ancestors · 2
Same direction, newest first
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
Other references · 13
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
- 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
- 2016Complex Embeddings for Simple Link Predictioncited by 902
- 2019RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Spacecited by 898
- 2019MMKG: Multi-Modal Knowledge Graphscited by 105
- 2016Image-embodied Knowledge Representation Learningcited by 68
- 2019Orthogonal Relation Transforms with Graph Context Modeling for Knowledge Graph Embeddingcited by 38
- 2023IMF: Interactive Multimodal Fusion Model for Link Predictioncited by 28
- 2022Knowledge Graph Completion with Pre-trained Multimodal Transformer and Twins Negative Samplingcited by 18
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025DiffusionCom: Structure-Aware Multimodal Diffusion Model for Multimodal Knowledge Graph Completion
- 2025Transformer-Based Multimodal Knowledge Graph Completion with Link-Aware Contexts
- 2025ELMM: Efficient Lightweight Multimodal Large Language Models for Multimodal Knowledge Graph Completion
- 2026RADD: Retrieval-Augmented Discrete Diffusion for Multi-Modal Knowledge Graph Completion
- 2025REMOTE: A Unified Multimodal Relation Extraction Framework with Multilevel Optimal Transport and Mixture-of-Experts
- 2025HERGC: Heterogeneous Experts Representation and Generative Completion for Multimodal Knowledge Graphs
- 2026Dual-Path LLM Reasoning for Multimodal Few-Shot Knowledge Graph Completion
- 2026Modality as Heterogeneity: Node Splitting and Graph Rewiring for Multimodal Graph Learning