mlsanityData to 14 Oct 2026

Noise-powered Multi-modal Knowledge Graph Representation Framework

arXiv 2403.06832 ↗First posted 11 Mar 2024cs.CL cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

The rise of Multi-modal Pre-training highlights the necessity for a unified Multi-Modal Knowledge Graph (MMKG) representation learning framework. Such a framework is essential for embedding structured knowledge into multi-modal Large Language Models effectively, alleviating issues like knowledge misconceptions and multi-modal hallucinations. In this work, we explore the efficacy of models in accurately embedding entities within MMKGs through two pivotal tasks: Multi-modal Knowledge Graph Completion (MKGC) and Multi-modal Entity Alignment (MMEA). Building on this foundation, we propose a novel SNAG method that utilizes a Transformer-based architecture equipped with modality-level noise masking to robustly integrate multi-modal entity features in KGs. By incorporating specific training objectives for both MKGC and MMEA, our approach achieves SOTA performance across a total of ten datasets, demonstrating its versatility. Moreover, SNAG can not only function as a standalone model but also enhance other existing methods, providing stable performance improvements. Code and data are available at https://github.com/zjukg/SNAG.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 34 of its 64 references that are in the map. First posted on arXiv 11 Mar 2024.

Builds on

34 of 64 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  8. 2017Graph Attention Networkscited by 8,923
  9. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  10. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  11. 2014Embedding Entities and Relations for Learning and Inference in Knowledge Basescited by 932
  12. 2019RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Spacecited by 898
22 more
  1. 2017Cross-lingual Entity Alignment via Joint Attribute-Preserving Embeddingcited by 127
  2. 2020A Benchmarking Study of Embedding-based Entity Alignment for Knowledge Graphscited by 120
  3. 2019MMKG: Multi-Modal Knowledge Graphscited by 105
  4. 2023NEFTune: Noisy Embeddings Improve Instruction Finetuningcited by 100
  5. 2022Multi-Modal Knowledge Graph Construction and Application: A Surveycited by 74
  6. 2023GraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graphcited by 69
  7. 2016Image-embodied Knowledge Representation Learningcited by 68
  8. 2024Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Surveycited by 53
  9. 2020Visual Pivoting for (Unsupervised) Entity Alignmentcited by 47
  10. 2019Aligning Cross-Lingual Entities with Multi-Aspect Informationcited by 46
  11. 2022Multi-modal Contrastive Representation Learning for Entity Alignmentcited by 38
  12. 2022Contrastive Language-Image Pre-Training with Knowledge Graphscited by 34
  13. 2023Rethinking Uncertainly Missing and Ambiguous Visual Modality in Multi-Modal Entity Alignmentcited by 21
  14. 2021Multi-modal Entity Alignment in Hyperbolic Spacecited by 19
  15. 2023Vision, Deduction and Alignment: An Empirical Study on Multi-modal Knowledge Graph Alignmentcited by 19
  16. 2022Knowledge Graph Completion with Pre-trained Multimodal Transformer and Twins Negative Samplingcited by 18
  17. 2022Endowing Language Models with Multimodal Knowledge Graph Representationscited by 11
  18. 2023Revisit and Outstrip Entity Alignment: A Perspective of Generative Modelscited by 8
  19. 2024Towards Semantic Consistency: Dirichlet Energy Driven Robust Multi-Modal Entity Alignmentcited by 8
  20. 2023Multi-Modal Knowledge Graph Transformer Framework for Multi-Modal Entity Alignmentcited by 7
  21. 2024Pseudo-Label Calibration Semi-supervised Multi-Modal Entity Alignmentcited by 6
  22. 2023End-to-End Learning on Multimodal Knowledge Graphscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026VL-KGE: Vision-Language Models Meet Knowledge Graph Embeddingscited by 2
  2. 2026MyGram: Modality-aware Graph Transformer with Global Distribution for Multi-modal Entity Alignmentcited by 0
  3. 2026Every Little Helps: Building Knowledge Graph Foundation Model with Fine-grained Transferable Multi-modal Tokenscited by 0
  4. 2026E-MMKGR: A Unified Multimodal Knowledge Graph Framework for E-commerce Applicationscited by 0
  5. 2026Are a Thousand Words Better Than a Single Picture? Beyond Images -- A Framework for Multi-Modal Knowledge Graph Dataset Enrichmentcited by 0
  6. 2026Time Imprint: Learning Time-Aware Representations in Multi-Modal Knowledge Graphscited by 0
  7. 2026More Perspectives, Stronger Signals: Multi-Perspective Enhancement and Progressive Fusion for Multimodal Entity Representation Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Transformer-Based Multimodal Knowledge Graph Completion with Link-Aware Contexts
  2. 2025ELMM: Efficient Lightweight Multimodal Large Language Models for Multimodal Knowledge Graph Completion
  3. 2025HERGC: Heterogeneous Experts Representation and Generative Completion for Multimodal Knowledge Graphs
  4. 2025Aligning Vision to Language: Annotation-Free Multimodal Knowledge Graph Construction for Enhanced LLMs Reasoning
  5. 2025ReaLM: Residual Quantization Bridging Knowledge Graph Embeddings and Large Language Models
  6. 2025MMGraphRAG: Bridging Vision and Language with Interpretable Multimodal Knowledge Graphs
  7. 2025A Zero-shot Learning Method Based on Large Language Models for Multi-modal Knowledge Graph Embedding
  8. 2023Making Large Language Models Perform Better in Knowledge Graph Completion