mlsanityData to 14 Oct 2026

MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts for Multimodal Knowledge Graph Completion

arXiv 2607.15592 ↗First posted 17 Jul 2026cs.AIcited by 0

On the map: Diffusion models → Image generation, text-to-image and editing

also in 1 more
Abstract (arXiv)

Multimodal Knowledge Graph Completion (MKGC) requires inferring missing entities from structural, textual, and visual cues. Existing diffusion-based MKGC methods usually denoise directly on raw multimodal features. Such a design forces the denoiser to simultaneously perform relation-dependent cue selection, cross-modal semantic alignment, and structure-aware entity generation, which introduces noisy and semantically inconsistent conditions for diffusion and consequently leads to suboptimal completion performance. To address this limitation, we propose MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts (MGDT), a novel MKGC framework built on an align-then-diffuse paradigm. MGDT first employs a Relation-Adaptive Semantic Routing Mixture-of-Experts (RASR-MoE) module to select relation-relevant multimodal semantic transformation paths and suppress irrelevant modality interference. MGDT then uses a frozen Multimodal Large Language Model (MLLM) as a semantic anchor to align the routed multimodal representations into a unified latent space and reduce cross-modal semantic heterogeneity. Finally, a Knowledge Graph Diffusion Transformer (KGDT) performs graph-conditioned denoising generation in the aligned space to produce the missing entity representation. Experiments on three benchmark datasets show that MGDT consistently outperforms strong baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 15 of its 28 references that are in the map. First posted on arXiv 17 Jul 2026.

Builds on

15 of 28 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 13

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  6. 2016Complex Embeddings for Simple Link Predictioncited by 902
  7. 2019RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Spacecited by 898
  8. 2019MMKG: Multi-Modal Knowledge Graphscited by 105
  9. 2016Image-embodied Knowledge Representation Learningcited by 68
  10. 2019Orthogonal Relation Transforms with Graph Context Modeling for Knowledge Graph Embeddingcited by 38
  11. 2023IMF: Interactive Multimodal Fusion Model for Link Predictioncited by 28
  12. 2022Knowledge Graph Completion with Pre-trained Multimodal Transformer and Twins Negative Samplingcited by 18
1 more
  1. 2024Tokenization, Fusion, and Augmentation: Towards Fine-grained Multi-modal Entity Representationcited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025DiffusionCom: Structure-Aware Multimodal Diffusion Model for Multimodal Knowledge Graph Completion
  2. 2025Transformer-Based Multimodal Knowledge Graph Completion with Link-Aware Contexts
  3. 2025ELMM: Efficient Lightweight Multimodal Large Language Models for Multimodal Knowledge Graph Completion
  4. 2026RADD: Retrieval-Augmented Discrete Diffusion for Multi-Modal Knowledge Graph Completion
  5. 2025REMOTE: A Unified Multimodal Relation Extraction Framework with Multilevel Optimal Transport and Mixture-of-Experts
  6. 2025HERGC: Heterogeneous Experts Representation and Generative Completion for Multimodal Knowledge Graphs
  7. 2026Dual-Path LLM Reasoning for Multimodal Few-Shot Knowledge Graph Completion
  8. 2026Modality as Heterogeneity: Node Splitting and Graph Rewiring for Multimodal Graph Learning