mlsanityData to 7 Oct 2026

Benchmarking and Improving Large Vision-Language Models for Fundamental Visual Graph Understanding and Reasoning

arXiv 2412.13540 ↗First posted 18 Dec 2024cs.CL cs.CVcited by 17

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Large Vision-Language Models (LVLMs) have demonstrated remarkable performance across diverse tasks. Despite great success, recent studies show that LVLMs encounter substantial limitations when engaging with visual graphs. To study the reason behind these limitations, we propose VGCure, a comprehensive benchmark covering 22 tasks for examining the fundamental graph understanding and reasoning capacities of LVLMs. Extensive evaluations conducted on 14 LVLMs reveal that LVLMs are weak in basic graph understanding and reasoning tasks, particularly those concerning relational or structurally complex information. Based on this observation, we propose a structure-aware fine-tuning framework to enhance LVLMs with structure learning abilities through three self-supervised learning tasks. Experiments validate the effectiveness of our method in improving LVLMs' performance on fundamental and downstream graph learning tasks, as well as enhancing their robustness against complex visual graphs.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 13 of its 50 references that are in the map. First posted on arXiv 18 Dec 2024.

Builds on

13 of 50 references.

References · 13

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  2. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  3. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  4. 2020TUDataset: A collection of benchmark datasets for learning with graphscited by 718
  5. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 617
  6. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
  7. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 285
  8. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 283
  9. 2020Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networkscited by 253
  10. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  11. 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoningcited by 83
  12. 2024GraphArena: Evaluating and Exploring Large Language Models on Graph Computationcited by 48
1 more
  1. 2024Large Language Models Are Cross-Lingual Knowledge-Free Reasonerscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 10.

17 papers.

Sort:

By area:

  1. 2025Evaluating and Steering Modality Preferences in Multimodal Large Language Modelcited by 33
  2. 2025Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1)cited by 23
  3. 2025Thinking in Character: Advancing Role-Playing Agents with Role-Aware Reasoningcited by 14
  4. 2026Graph is a Substrate Across Data Modalitiescited by 13
  5. 2025Memory-augmented Query Reconstruction for LLM-based Knowledge Graph Reasoningcited by 10
  6. 2025LoCoT2V-Bench: Benchmarking Long-Form and Complex Text-to-Video Generationcited by 6
  7. 2026Instruction Anchor: Dissecting the Mechanistic Dynamics of Modality Arbitrationcited by 6
  8. 2025Beyond the Vision Encoder: Identifying and Mitigating Spatial Bias in Large Vision-Language Modelscited by 2
  9. 2025See or Say Graphs: Agent-Driven Scalable Graph Structure Understanding with Vision-Language Modelscited by 2
  10. 2026PPU-Bench:Real World Benchmark for Personalized Partial Unlearning in Vision Language Modelscited by 1
  11. 2026Beyond Token-Level Policy Gradients for Complex Reasoning with Large Language Modelscited by 0
  12. 2026Nodes Are Early, Edges Are Late: Probing Diagram Representations in Large Vision-Language Modelscited by 0
5 more
  1. 2026Are Large Language Models Suitable for Graph Computation? Progress and Prospectscited by 0
  2. 2026Prior Bias in Vision Language Models on UML Diagram Interpretationcited by 0
  3. 2026A Systematic Survey on Image Description Techniques for STEM Domainscited by 0
  4. 2026RepoAtlas: Guiding Coding Agents via Evolving Multimodal Repository Viewscited by 0
  5. 2026VisKG-LM: Compiling Knowledge Graphs into Visual Memory for Multiple-Choice Question Answeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Graph-to-Vision: Multi-graph Understanding and Reasoning using Vision-Language Models
  2. 2026GraphVerse: A Comprehensive Visual Graph Reasoning Benchmark for Multimodal Large Language Models
  3. 2026HyperGVL: Benchmarking and Improving Large Vision-Language Models in Hypergraph Understanding and Reasoning
  4. 2024VisGraphVar: A Benchmark Generator for Assessing Variability in Graph Analysis Using Large Vision-Language Models
  5. 2026GraphVLM: Benchmarking Vision Language Models for Multimodal Graph Learning
  6. 2026SmoGVLM: A Small, Graph-enhanced Vision-Language Model
  7. 2026Scene Graph Thinking: Reinforcing Structured Visual Reasoning for Multimodal Large Language Models
  8. 2024GITA: Graph to Visual and Textual Integration for Vision-Language Graph Reasoning