# FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts

arXiv: https://arxiv.org/abs/2406.19237 · HTML: https://mlsanity.com/paper/2406.19237/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 22 of its 40 references that are in the map.
- First posted on arXiv 27 Jun 2024.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 22 of 22)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality](https://mlsanity.com/paper/2304.14178/) (2023)
- [Multimodal Chain-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2302.00923/) (2023)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [CogAgent: A Visual Language Model for GUI Agents](https://mlsanity.com/paper/2312.08914/) (2023)
- [From Recognition to Cognition: Visual Commonsense Reasoning](https://mlsanity.com/paper/1811.10830/) (2018)
- [InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model](https://mlsanity.com/paper/2401.16420/) (2024)
- [PlotQA: Reasoning over Scientific Plots](https://mlsanity.com/paper/1909.00997/) (2019)
- [A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models](https://mlsanity.com/paper/2303.10420/) (2023)
- [ChartLlama: A Multimodal LLM for Chart Understanding and Generation](https://mlsanity.com/paper/2311.16483/) (2023)
- [ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoning](https://mlsanity.com/paper/2402.12185/) (2024)
- [ChartBench: A Benchmark for Complex Visual Reasoning in Charts](https://mlsanity.com/paper/2312.15915/) (2023)
- [Guiding Large Language Models via Directional Stimulus Prompting](https://mlsanity.com/paper/2302.11520/) (2023)
- [Measuring Faithful and Plausible Visual Grounding in VQA](https://mlsanity.com/paper/2305.15015/) (2023)

## Built on by (most cited first, 28 of 28)

- [Baichuan-Omni-1.5 Technical Report](https://mlsanity.com/paper/2501.15368/) (2025)
- [Do Vision-Language Models Really Understand Visual Language?](https://mlsanity.com/paper/2410.00193/) (2024)
- [Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Model](https://mlsanity.com/paper/2407.07053/) (2024)
- [Beyond End-to-End VLMs: Leveraging Intermediate Text Representations for Superior Flowchart Understanding](https://mlsanity.com/paper/2412.16420/) (2024)
- [Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence](https://mlsanity.com/paper/2606.15932/) (2026)
- [Follow the Flow: Fine-grained Flowchart Attribution with Neurosymbolic Agents](https://mlsanity.com/paper/2506.01344/) (2025)
- [From Pixels to Paths: A Multi-Agent Framework for Editable Scientific Illustration](https://mlsanity.com/paper/2510.27452/) (2025)
- [Flowchart2Mermaid: A Vision-Language Model Powered System for Converting Flowcharts into Editable Diagram Code](https://mlsanity.com/paper/2512.02170/) (2025)
- [OmniDiagram: Advancing Unified Diagram Code Generation via Visual Interrogation Reward](https://mlsanity.com/paper/2604.05514/) (2026)
- [Multilingual Multimodal Software Developer for Code Generation](https://mlsanity.com/paper/2507.08719/) (2025)
- [A Graph-based Approach for Multi-Modal Question Answering from Flowcharts in Telecom Documents](https://mlsanity.com/paper/2507.22938/) (2025)
- [Structured Extraction from Business Process Diagrams Using Vision-Language Models](https://mlsanity.com/paper/2511.22448/) (2025)
- [TechING: Towards Real World Technical Image Understanding via VLMs](https://mlsanity.com/paper/2601.18238/) (2026)
- [Exploring Primitive Visual Measurement Understanding and the Role of Output Format in Learning in Vision-Language Models](https://mlsanity.com/paper/2501.15144/) (2025)
- [Structure-aware Contrastive Learning for Diagram Understanding of Multimodal Models](https://mlsanity.com/paper/2509.01959/) (2025)
- [Beyond Superficial Unlearning: Sharpness-Aware Robust Erasure of Hallucinations in Multimodal LLMs](https://mlsanity.com/paper/2601.16527/) (2026)
- [Multimodal Information Fusion for Chart Understanding: A Survey of MLLMs -- Evolution, Limitations, and Cognitive Enhancement](https://mlsanity.com/paper/2602.10138/) (2026)
- [EdgeFlow: Edge-Map Augmented VLM-Based Flowchart Processing for Industrial Requirements Engineering](https://mlsanity.com/paper/2605.27332/) (2026)
- [Enginuity: A Dataset and Benchmark for Vision-Language Understanding of Engineering Diagrams](https://mlsanity.com/paper/2606.03410/) (2026)
- [PPTBench: Can Coding Agents Reconstruct the Visual World through Structured, Editable Slides](https://mlsanity.com/paper/2609.29718/) (2026)
- [Can Visual Encoder Learn to See Arrows?](https://mlsanity.com/paper/2505.19944/) (2025)
- [JSynFlow: Japanese Synthesised Flowchart Visual Question Answering Dataset built with Large Language Models](https://mlsanity.com/paper/2602.04142/) (2026)
- [An Online Reference-Free Evaluation Framework for Flowchart Image-to-Code Generation](https://mlsanity.com/paper/2602.13376/) (2026)
- [FloCA: Towards Faithful and Logically Consistent Flowchart Reasoning](https://mlsanity.com/paper/2602.14035/) (2026)
- [Pseudo Contrastive Learning for Diagram Comprehension in Multimodal Models](https://mlsanity.com/paper/2602.23589/) (2026)
- [StructBreak: Structural Cognitive Overload-Induced Safety Failures in MLLMs](https://mlsanity.com/paper/2605.25534/) (2026)
- [Running the Gauntlet: Challenging Agentic Tasks](https://mlsanity.com/paper/2606.14397/) (2026)
- [ComplexityWorld: Benchmarking Vision-Language Models on Verifiable Visual Decision Making](https://mlsanity.com/paper/2608.07584/) (2026)

## Abstract (arXiv)

Existing benchmarks for visual question answering lack in visual grounding and complexity, particularly in evaluating spatial reasoning skills. We introduce FlowVQA, a novel benchmark aimed at assessing the capabilities of visual question-answering multimodal language models in reasoning with flowcharts as visual contexts. FlowVQA comprises 2,272 carefully generated and human-verified flowchart images from three distinct content sources, along with 22,413 diverse question-answer pairs, to test a spectrum of reasoning tasks, including information localization, decision-making, and logical progression. We conduct a thorough baseline evaluation on a suite of both open-source and proprietary multimodal language models using various strategies, followed by an analysis of directional bias. The results underscore the benchmark's potential as a vital tool for advancing the field of multimodal modeling, providing a focused and challenging environment for enhancing model performance in visual and logical reasoning tasks.