mlsanityData to 7 Oct 2026

MMCOMET: A Large-Scale Multimodal Commonsense Knowledge Graph for Contextual Reasoning

arXiv 2603.01055 ↗First posted 1 Mar 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We present MMCOMET, the first multimodal commonsense knowledge graph (MMKG) that integrates physical, social, and eventive knowledge. MMCOMET extends the ATOMIC2020 knowledge graph to include a visual dimension, through an efficient image retrieval process, resulting in over 900K multimodal triples. This new resource addresses a major limitation of existing MMKGs in supporting complex reasoning tasks like image captioning and storytelling. Through a standard visual storytelling experiment, we show that our holistic approach enables the generation of richer, coherent, and contextually grounded stories than those produced using text-only knowledge. This resource establishes a new foundation for multimodal commonsense reasoning and narrative generation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 48 references that are in the map. First posted on arXiv 1 Mar 2026.

Builds on

22 of 48 references.

References · 22

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  3. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  4. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  5. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  6. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  7. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  8. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
  9. 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 415
  10. 2020COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphscited by 254
  11. 2019MMKG: Multi-Modal Knowledge Graphscited by 104
  12. 2022Extracting Cultural Commonsense Knowledge at Scalecited by 76
10 more
  1. 2020TransOMCS: From Linguistic Graphs to Commonsense Knowledgecited by 52
  2. 2018A Survey on Open Information Extractioncited by 51
  3. 2020CSKG: The CommonSense Knowledge Graphcited by 45
  4. 2020VisualSem: A High-quality Knowledge Graph for Vision and Languagecited by 30
  5. 2021"I'm Not Mad": Commonsense Implications of Negation and Contradictioncited by 26
  6. 2021Refined Commonsense Knowledge from Large-Scale Web Contentscited by 19
  7. 2019Improving Image Captioning by Leveraging Knowledge Graphscited by 11
  8. 2022RoViST:Learning Robust Metrics for Visual Storytellingcited by 7
  9. 2023Completeness, Recall, and Negation in Open-World Knowledge Bases: A Surveycited by 7
  10. 2024SCO-VIST: Social Interaction Commonsense Knowledge-based Visual Storytellingcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025VAT-KG: Knowledge-Intensive Multimodal Knowledge Graph Dataset for Retrieval-Augmented Generation
  2. 2025Hierarchical Knowledge Graphs for Story Understanding in Visual Narratives
  3. 2024Multimodal Reasoning with Multimodal Knowledge Graph
  4. 2025MMGraphRAG: Bridging Vision and Language with Interpretable Multimodal Knowledge Graphs
  5. 2025MMMG: A Massive, Multidisciplinary, Multi-Tier Generation Benchmark for Text-to-Image Reasoning
  6. 2025Taming the Untamed: Graph-Based Knowledge Retrieval and Reasoning for MLLMs to Conquer the Unknown
  7. 2025VisualQuest: A Benchmark for Abstract Visual Reasoning in MLLMs
  8. 2022Reasoning with Multi-Structure Commonsense Knowledge in Visual Dialog