mlsanityData to 7 Oct 2026

VL-KGE: Vision-Language Models Meet Knowledge Graph Embeddings

arXiv 2603.02435 ↗First posted 2 Mar 2026cs.AI cs.LGcited by 2

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Real-world multimodal knowledge graphs (MKGs) are inherently heterogeneous, modeling entities that are associated with diverse modalities. Traditional knowledge graph embedding (KGE) methods excel at learning continuous representations of entities and relations, yet they are typically designed for unimodal settings. Recent approaches extend KGE to multimodal settings but remain constrained, often processing modalities in isolation, resulting in weak cross-modal alignment, and relying on simplistic assumptions such as uniform modality availability across entities. Vision-Language Models (VLMs) offer a powerful way to align diverse modalities within a shared embedding space. We propose Vision-Language Knowledge Graph Embeddings (VL-KGE), a framework that integrates cross-modal alignment from VLMs with structured relational modeling to learn unified multimodal representations of knowledge graphs. Experiments on WN9-IMG and two novel fine art MKGs, WikiArt-MKG-v1 and WikiArt-MKG-v2, demonstrate that VL-KGE consistently improves over traditional unimodal and multimodal KGE methods in link prediction tasks. Our results highlight the value of VLMs for multimodal KGE, enabling more robust and structured reasoning over large-scale heterogeneous knowledge graphs.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 85 references that are in the map. First posted on arXiv 2 Mar 2026.

Builds on

45 of 85 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023UKnow: A Unified Knowledge Protocol with Multimodal Knowledge Graph Datasets for Reasoning and Vision-Language Pre-Training14 Feb
  2. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervision24 Aug

Other references · 43

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2017Inductive Representation Learning on Large Graphscited by 6,284
  6. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  7. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  8. 2015VQA: Visual Question Answeringcited by 2,489
  9. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  10. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  11. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  12. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
31 more
  1. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
  2. 2014Embedding Entities and Relations for Learning and Inference in Knowledge Basescited by 929
  3. 2016Complex Embeddings for Simple Link Predictioncited by 897
  4. 2019RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Spacecited by 893
  5. 2017Convolutional 2D Knowledge Graph Embeddingscited by 729
  6. 2020TUDataset: A collection of benchmark datasets for learning with graphscited by 718
  7. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  8. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 328
  9. 2021UniT: Multimodal Multitask Learning with a Unified Transformercited by 195
  10. 2021VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioningcited by 190
  11. 2017A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Networkcited by 143
  12. 2019MMKG: Multi-Modal Knowledge Graphscited by 104
  13. 2023UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Webcited by 77
  14. 2021NodePiece: Compositional and Parameter-Efficient Representations of Large Knowledge Graphscited by 71
  15. 2016Image-embodied Knowledge Representation Learningcited by 68
  16. 2025UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphscited by 47
  17. 2017BAM! The Behance Artistic Media Dataset for Recognition Beyond Photographycited by 43
  18. 2024Multimodal Reasoning with Multimodal Knowledge Graphcited by 43
  19. 2023Heterogeneous Federated Knowledge Graph Embedding Learning and Unlearningcited by 39
  20. 2022It is Okay to Not Be Okay: Overcoming Emotional Bias in Affective Image Captioning by Contrastive Data Collectioncited by 36
  21. 2022Contrastive Language-Image Pre-Training with Knowledge Graphscited by 34
  22. 2018The Shape of Art History in the Eyes of the Machinecited by 33
  23. 2021Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generationcited by 30
  24. 2021Understanding the Performance of Knowledge Graph Embeddings in Drug Discoverycited by 18
  25. 2022MoSE: Modality Split and Ensemble for Multimodal Knowledge Graph Completioncited by 14
  26. 2021GCNBoost: Artwork Classification by Label Propagation through a Knowledge Graphcited by 11
  27. 2021Graph Neural Networks for Knowledge Enhanced Visual Representation of Paintingscited by 9
  28. 2024IME: Integrating Multi-curvature Shared and Specific Embedding for Temporal Knowledge Graph Completioncited by 9
  29. 2024Noise-powered Multi-modal Knowledge Graph Representation Frameworkcited by 7
  30. 2025ArtRAG: Retrieval-Augmented Generation with Structured Context for Visual Art Understandingcited by 7
  31. 2024ReliK: A Reliability Measure for Knowledge Graph Embeddingscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026A-MAR: Agent-based Multimodal Art Retrieval for Fine-Grained Artwork Understandingcited by 1
  2. 2026MMArt: A Multi-Perspective Multimodal Dataset for Visual Art Understandingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Aligning Vision to Language: Annotation-Free Multimodal Knowledge Graph Construction for Enhanced LLMs Reasoning
  2. 2025Transformer-Based Multimodal Knowledge Graph Completion with Link-Aware Contexts
  3. 2025MMGraphRAG: Bridging Vision and Language with Interpretable Multimodal Knowledge Graphs
  4. 2026GraphVLM: Benchmarking Vision Language Models for Multimodal Graph Learning
  5. 2026SmoGVLM: A Small, Graph-enhanced Vision-Language Model
  6. 2026ViSR-KGC: Visual Subgraph Reasoning with Vision-Language Models for Multimodal Knowledge Graph Completion
  7. 2026One Model, Many Graphs: Learning over Attributed Graphs across Heterogeneous Modalities with Vision-Language Models
  8. 2025ELMM: Efficient Lightweight Multimodal Large Language Models for Multimodal Knowledge Graph Completion