mlsanityData to 7 Oct 2026

Multimodal Analogical Reasoning over Knowledge Graphs

arXiv 2210.00312 ↗First posted 1 Oct 2022cs.CL cs.AI cs.CVcited by 21

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Analogical reasoning is fundamental to human cognition and holds an important place in various fields. However, previous studies mainly focus on single-modal analogical reasoning and ignore taking advantage of structure knowledge. Notably, the research in cognitive psychology has demonstrated that information from multimodal sources always brings more powerful cognitive transfer than single modality sources. To this end, we introduce the new task of multimodal analogical reasoning over knowledge graphs, which requires multimodal reasoning ability with the help of background knowledge. Specifically, we construct a Multimodal Analogical Reasoning dataSet (MARS) and a multimodal knowledge graph MarKG. We evaluate with multimodal knowledge graph embedding and pre-trained Transformer baselines, illustrating the potential challenges of the proposed task. We further propose a novel model-agnostic Multimodal analogical reasoning framework with Transformer (MarT) motivated by the structure mapping theory, which can obtain better performance. Code and datasets are available in https://github.com/zjunlp/MKG_Analogy.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 46 references that are in the map. First posted on arXiv 1 Oct 2022.

Builds on

28 of 46 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices28 Jan

Other references · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  5. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  6. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  7. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  8. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  9. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  10. 2016Complex Embeddings for Simple Link Predictioncited by 897
  11. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  12. 2019What Makes Training Multi-Modal Classification Networks Hard?cited by 340
15 more
  1. 2022Balanced Multimodal Learning via On-the-fly Gradient Modulationcited by 259
  2. 2019RAVEN: A Dataset for Relational and Analogical Visual rEasoNingcited by 225
  3. 2018Measuring abstract reasoning in neural networkscited by 165
  4. 2019MMKG: Multi-Modal Knowledge Graphscited by 104
  5. 2022Multi-Modal Knowledge Graph Construction and Application: A Surveycited by 74
  6. 2020Stratified Rule-Aware Network for Abstract Visual Reasoningcited by 72
  7. 2017Analogical Inference for Multi-Relational Embeddingscited by 69
  8. 2016Image-embodied Knowledge Representation Learningcited by 68
  9. 2018Towards Understanding Linear Word Analogiescited by 57
  10. 2021BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?cited by 56
  11. 2019Learning to Make Analogies by Contrasting Abstract Relational Structurecited by 53
  12. 2022Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completioncited by 44
  13. 2022E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoningcited by 28
  14. 2021Selective Replay Enhances Learning in Online Continual Analogical Reasoningcited by 12
  15. 2020Analogical Reasoning for Visually Grounded Language Acquisitioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 6; 2024: 5; 2025: 6; 2026: 2.

21 papers.

Sort:

By area:

  1. 2022Reasoning with Language Model Prompting: A Surveycited by 251
  2. 2022A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodalcited by 106
  3. 2024Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Modelscited by 91
  4. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 89
  5. 2024Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Surveycited by 50
  6. 2023Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Modelscited by 37
  7. 2024Learning on Multimodal Graphs: A Surveycited by 22
  8. 2025Mind with Eyes: from Language Reasoning to Multimodal Reasoningcited by 22
  9. 2025Graph4MM: Weaving Multimodal Learning with Structural Informationcited by 17
  10. 2023ANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Basecited by 16
  11. 2023Continual Multimodal Knowledge Graph Constructioncited by 15
  12. 2023UKnow: A Unified Knowledge Protocol with Multimodal Knowledge Graph Datasets for Reasoning and Vision-Language Pre-Trainingcited by 14
9 more
  1. 2023Beneath Surface Similarity: Large Language Models Make Reasonable Scientific Analogies after Structure Abductioncited by 11
  2. 2024Applications of Generative AI (GAI) for Mobile and Wireless Networking: A Surveycited by 9
  3. 2023Knowledge Rumination for Pre-trained Language Modelscited by 2
  4. 2024MolMetaLM: a Physicochemical Knowledge-Guided Molecular Meta Language Modelcited by 2
  5. 2025VAT-KG: Knowledge-Intensive Multimodal Knowledge Graph Dataset for Retrieval-Augmented Generationcited by 2
  6. 2026CARV: A Diagnostic Benchmark for Compositional Analogical Reasoning in Multimodal LLMscited by 1
  7. 2025Towards Multimodal Graph Large Language Modelcited by 0
  8. 2025Retrieval-augmented Prompt Learning for Pre-trained Foundation Modelscited by 0
  9. 2026PRISM: A Category-Theoretic Framework for Measuring and Refining Multimodal Analogiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Multimodal Reasoning with Multimodal Knowledge Graph
  2. 2024Can Multimodal Large Language Model Think Analogically?
  3. 2025Enhancing multimodal analogical reasoning with Logic Augmented Generation
  4. 2023Structure Guided Multi-modal Pre-trained Transformer for Knowledge Graph Reasoning
  5. 2024LLMs as Models for Analogical Reasoning
  6. 2022DeepGAR: Deep Graph Learning for Analogical Reasoning
  7. 2026Mario: Multimodal Graph Reasoning with Large Language Models
  8. 2022Emergent Analogical Reasoning in Large Language Models