mlsanityData to 7 Oct 2026

MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering

arXiv 2503.18491 ↗First posted 24 Mar 2025cs.CLcited by 18

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Visual Question Answering (VQA) requires reasoning across visual and textual modalities, yet Large Vision-Language Models (LVLMs) often lack integrated commonsense knowledge, limiting their robustness in real-world scenarios. To address this, we introduce MAGIC-VQA, a novel framework that enhances VQA by systematically integrating commonsense knowledge with LVLMs. MAGIC-VQA employs a three-stage process: (1) Explicit Knowledge Integration from external sources, (2) By-Type Post-Processing for contextual refinement, and (3) Implicit Knowledge Augmentation using a Graph Neural Network (GNN) for structured reasoning. While GNNs bring greater depth to structured inference, they enable superior relational inference beyond LVLMs. MAGIC-VQA bridges a key gap by unifying commonsensse knowledge with LVLM-driven reasoning, eliminating the need for extensive pre-training or complex prompt tuning. Our framework achieves state-of-the-art performance on benchmark datasets, significantly improving commonsense reasoning in VQA.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 42 of its 64 references that are in the map. First posted on arXiv 24 Mar 2025.

Builds on

42 of 64 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024MARVEL: Multidimensional Abstraction and Reasoning through Visual Evaluation and Learning21 Apr
  2. 2023ROME: Evaluating Pre-trained Vision-Language Models on Reasoning beyond Visual Common Sense30 Oct
  3. 2022Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles29 Nov
  4. 2022Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation12 Mar

Other references · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  7. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  8. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  9. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  10. 2015VQA: Visual Question Answeringcited by 2,489
  11. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  12. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
26 more
  1. 2019Towards VQA Models That Can Readcited by 1,696
  2. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  3. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  4. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  5. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  6. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  7. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  8. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  9. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  10. 2020COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphscited by 254
  11. 2021KAT: A Knowledge Augmented Transformer for Vision-and-Languagecited by 125
  12. 2020KRISP: Integrating Implicit and Symbolic Knowledge for Open-Domain Knowledge-Based VQAcited by 124
  13. 2022REVEAL: Retrieval-Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowledge Memorycited by 122
  14. 2022Retrieval Augmented Visual Question Answering with Outside Knowledgecited by 93
  15. 2021Multi-Modal Answer Validation for Knowledge-Based VQAcited by 78
  16. 2024KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts Reasoningcited by 60
  17. 2023T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Mixed Large Language Model Signals for Science Question Answeringcited by 59
  18. 2025PRISM: Self-Pruning Intrinsic Selection Method for Training-Free Multimodal Data Selectioncited by 57
  19. 2022Vision-and-Language Pretrained Models: A Surveycited by 50
  20. 2024Core Knowledge Deficits in Multi-Modal Language Modelscited by 38
  21. 2025HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden Statescited by 29
  22. 2020VICTR: Visual Information Captured Text Representation for Text-to-Image Multimodal Taskscited by 12
  23. 2023Localized Symbolic Knowledge Distillation for Visual Commonsense Modelscited by 12
  24. 2025Debt Collection Negotiations with Large Language Models: An Evaluation System and Optimizing Decision Making with Multi-Agentcited by 9
  25. 2024SSFold: Learning to Fold Arbitrary Crumpled Cloth Using Graph Dynamics from Human Demonstrationcited by 7
  26. 2024Do Vision-Language Transformers Exhibit Visual Commonsense? An Empirical Study of VCRcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 11; 2026: 7.

18 papers.

Sort:

By area:

  1. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 89
  2. 2025EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewardscited by 24
  3. 2025Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsingcited by 17
  4. 2025Eigenspectrum Analysis of Neural Networks without Aspect Ratio Biascited by 9
  5. 2026Omni2Sound: Towards Unified Video-Text-to-Audio Generationcited by 8
  6. 2026SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction Tuningcited by 8
  7. 2025Diagnosing Visual Reasoning: Challenges, Insights, and a Path Forwardcited by 6
  8. 2026TAG-MoE: Task-Aware Gating for Unified Generative Mixture-of-Expertscited by 6
  9. 2025A Narrative Review on Large AI Models in Lung Cancer Screening, Diagnosis, and Treatment Planningcited by 4
  10. 2025Cognitive-Inspired Hierarchical Attention Fusion With Visual and Textual for Cross-Domain Sequential Recommendationcited by 2
  11. 2025Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsingcited by 1
  12. 2025VisionTrap: Unanswerable Questions On Visual Datacited by 1
6 more
  1. 2026RCP: Representation Consistency Pruner for Mitigating Distribution Shift in Large Vision-Language Modelscited by 1
  2. 2026From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMscited by 1
  3. 2026ToolAnchor: Anchoring Counterfactual Context to Boost Agentic Tool-use Capabilitycited by 1
  4. 2026Semiotic logical hexagon theory for LLM logical reasoningcited by 1
  5. 2025A Multi-Modal Neuro-Symbolic Approach for Spatial Reasoning-Based Visual Grounding in Roboticscited by 0
  6. 2025MIND: Multi-rationale INtegrated Discriminative Reasoning Framework for Multi-modal Large Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey
  2. 2025NLKI: A lightweight Natural Language Knowledge Integration Framework for Improving Small VLMs in Commonsense VQA Tasks
  3. 2026MaS-VQA: A Mask-and-Select Framework for Knowledge-Based Visual Question Answering
  4. 2025Elevating Visual Question Answering through Implicitly Learned Reasoning Pathways in LVLMs
  5. 2026Enhancing Visual Question Answering with Multimodal LLMs via Chain-of-Question Guided Retrieval-Augmented Generation
  6. 2026KG-ViP: Bridging Knowledge Grounding and Visual Perception in Multi-modal LLMs for Visual Question Answering
  7. 2025StaR-KVQA: Structured Reasoning Traces for Implicit-Knowledge Visual Question Answering
  8. 2024Multimodal Commonsense Knowledge Distillation for Visual Question Answering