mlsanityData to 14 Oct 2026

SmoGVLM: A Small, Graph-enhanced Vision-Language Model

arXiv 2604.16517 ↗First posted 15 Apr 2026cs.CV cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Large vision-language models (VLMs) achieve strong performance on multimodal tasks but often suffer from hallucination and poor grounding in knowledge-intensive reasoning. We propose SmoGVLM, a small, graph-enhanced VLM that integrates structured knowledge with visual and textual modalities, using Graph Neural Networks. We investigate the effects of our method across a range of model sizes, from tiny (1.3B) to large (13B) models. Our results demonstrate that, when trained using our approach, a small model can achieve performance gains upto 16.24%, and surpass its larger counterparts, outperforming larger VLMs and strong fine-tuned baselines. These results highlight the potential of structured knowledge augmentation for efficient, smaller-scale multimodal reasoning systems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 20 references that are in the map. First posted on arXiv 15 Apr 2026.

Builds on

18 of 20 references.

References · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  7. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  8. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  9. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,694
  10. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
  11. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  12. 2021QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answeringcited by 388
6 more
  1. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 357
  2. 2019KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoningcited by 216
  3. 2023Knowledge Editing for Large Language Models: A Surveycited by 194
  4. 2022Deep Bidirectional Language-Knowledge Graph Pretrainingcited by 156
  5. 2019Relational Graph Attention Networkscited by 82
  6. 2024KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts Reasoningcited by 60

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026GraphVLM: Benchmarking Vision Language Models for Multimodal Graph Learning
  2. 2025Dynamic Knowledge Integration for Enhanced Vision-Language Reasoning
  3. 2024Benchmarking and Improving Large Vision-Language Models for Fundamental Visual Graph Understanding and Reasoning
  4. 2024NPHardEval4V: Dynamic Evaluation of Large Vision-Language Models with Effects of Vision
  5. 2026VL-KGE: Vision-Language Models Meet Knowledge Graph Embeddings
  6. 2024Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models
  7. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions
  8. 2026Scene Graph Thinking: Reinforcing Structured Visual Reasoning for Multimodal Large Language Models