mlsanityData to 7 Oct 2026

FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts

arXiv 2406.19237 ↗First posted 27 Jun 2024cs.CL cs.CV cs.IRcited by 27

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Existing benchmarks for visual question answering lack in visual grounding and complexity, particularly in evaluating spatial reasoning skills. We introduce FlowVQA, a novel benchmark aimed at assessing the capabilities of visual question-answering multimodal language models in reasoning with flowcharts as visual contexts. FlowVQA comprises 2,272 carefully generated and human-verified flowchart images from three distinct content sources, along with 22,413 diverse question-answer pairs, to test a spectrum of reasoning tasks, including information localization, decision-making, and logical progression. We conduct a thorough baseline evaluation on a suite of both open-source and proprietary multimodal language models using various strategies, followed by an analysis of directional bias. The results underscore the benchmark's potential as a vital tool for advancing the field of multimodal modeling, providing a focused and challenging environment for enhancing model performance in visual and logical reasoning tasks.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 22 of its 40 references that are in the map. First posted on arXiv 27 Jun 2024.

Builds on

22 of 40 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoning19 Feb
  2. 2023ChartBench: A Benchmark for Complex Visual Reasoning in Charts26 Dec
  3. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  4. 2023Multimodal Chain-of-Thought Reasoning in Language Models2 Feb
  5. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering20 Sep

Other references · 17

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  4. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  5. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  6. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  7. 2019Towards VQA Models That Can Readcited by 1,696
  8. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  9. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  10. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  11. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  12. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
5 more
  1. 2019PlotQA: Reasoning over Scientific Plotscited by 317
  2. 2023A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Modelscited by 244
  3. 2023ChartLlama: A Multimodal LLM for Chart Understanding and Generationcited by 158
  4. 2023Guiding Large Language Models via Directional Stimulus Promptingcited by 94
  5. 2023Measuring Faithful and Plausible Visual Grounding in VQAcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 10; 2026: 14.

27 papers.

Sort:

By area:

  1. 2025Baichuan-Omni-1.5 Technical Reportcited by 107
  2. 2024Do Vision-Language Models Really Understand Visual Language?cited by 29
  3. 2024Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Modelcited by 21
  4. 2024Beyond End-to-End VLMs: Leveraging Intermediate Text Representations for Superior Flowchart Understandingcited by 10
  5. 2025Follow the Flow: Fine-grained Flowchart Attribution with Neurosymbolic Agentscited by 4
  6. 2025From Pixels to Paths: A Multi-Agent Framework for Editable Scientific Illustrationcited by 4
  7. 2025A Graph-based Approach for Multi-Modal Question Answering from Flowcharts in Telecom Documentscited by 3
  8. 2025Structured Extraction from Business Process Diagrams Using Vision-Language Modelscited by 3
  9. 2025Flowchart2Mermaid: A Vision-Language Model Powered System for Converting Flowcharts into Editable Diagram Codecited by 3
  10. 2026OmniDiagram: Advancing Unified Diagram Code Generation via Visual Interrogation Rewardcited by 3
  11. 2025Multilingual Multimodal Software Developer for Code Generationcited by 2
  12. 2026TechING: Towards Real World Technical Image Understanding via VLMscited by 2
15 more
  1. 2025Exploring Primitive Visual Measurement Understanding and the Role of Output Format in Learning in Vision-Language Modelscited by 1
  2. 2025Structure-aware Contrastive Learning for Diagram Understanding of Multimodal Modelscited by 1
  3. 2026Beyond Superficial Unlearning: Sharpness-Aware Robust Erasure of Hallucinations in Multimodal LLMscited by 1
  4. 2026Multimodal Information Fusion for Chart Understanding: A Survey of MLLMs -- Evolution, Limitations, and Cognitive Enhancementcited by 1
  5. 2025Can Visual Encoder Learn to See Arrows?cited by 0
  6. 2026JSynFlow: Japanese Synthesised Flowchart Visual Question Answering Dataset built with Large Language Modelscited by 0
  7. 2026An Online Reference-Free Evaluation Framework for Flowchart Image-to-Code Generationcited by 0
  8. 2026FloCA: Towards Faithful and Logically Consistent Flowchart Reasoningcited by 0
  9. 2026Pseudo Contrastive Learning for Diagram Comprehension in Multimodal Modelscited by 0
  10. 2026StructBreak: Structural Cognitive Overload-Induced Safety Failures in MLLMscited by 0
  11. 2026EdgeFlow: Edge-Map Augmented VLM-Based Flowchart Processing for Industrial Requirements Engineeringcited by 0
  12. 2026Enginuity: A Dataset and Benchmark for Vision-Language Understanding of Engineering Diagramscited by 0
  13. 2026Running the Gauntlet: Challenging Agentic Taskscited by 0
  14. 2026ComplexityWorld: Benchmarking Vision-Language Models on Verifiable Visual Decision Makingcited by 0
  15. 2026PPTBench: Can Coding Agents Reconstruct the Visual World through Structured, Editable Slidescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ReasonMap: Towards Fine-Grained Visual Reasoning from Transit Maps
  2. 2024MultiChartQA: Benchmarking Vision-Language Models on Multi-Chart Problems
  3. 2026VISTAQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidence
  4. 2025ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models
  5. 2025ChartMind: A Comprehensive Benchmark for Complex Real-world Multimodal Chart Question Answering
  6. 2024A Comprehensive Survey on Visual Question Answering Datasets and Algorithms
  7. 2025Exploring Advanced Techniques for Visual Question Answering: A Comprehensive Comparison
  8. 2025MTabVQA: Evaluating Multi-Tabular Reasoning of Language Models in Visual Space