# Answering Questions about Data Visualizations using Efficient Bimodal Fusion

arXiv: https://arxiv.org/abs/1908.01801 · HTML: https://mlsanity.com/paper/1908.01801/

## Key facts

- Cited by 26 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 5 Aug 2019.

## Built on by (most cited first, 26 of 26)

- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [MapQA: A Dataset for Question Answering on Choropleth Maps](https://mlsanity.com/paper/2211.08545/) (2022)
- [From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Models](https://mlsanity.com/paper/2403.12027/) (2024)
- [Reading and Reasoning over Chart Images for Evidence-based Automated Fact-Checking](https://mlsanity.com/paper/2301.11843/) (2023)
- [EvoChart: A Benchmark and a Self-Training Approach Towards Real-World Chart Understanding](https://mlsanity.com/paper/2409.01577/) (2024)
- [Chart Question Answering: State of the Art and Future Directions](https://mlsanity.com/paper/2205.03966/) (2022)
- [ChartCheck: Explainable Fact-Checking over Real-World Chart Images](https://mlsanity.com/paper/2311.07453/) (2023)
- [ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules](https://mlsanity.com/paper/2304.02173/) (2023)
- [Visual Question Answering: A Survey on Techniques and Common Trends in Recent Literature](https://mlsanity.com/paper/2305.11033/) (2023)
- [LineCap: Line Charts for Data Visualization Captioning Models](https://mlsanity.com/paper/2207.07243/) (2022)
- [Enhanced Chart Understanding in Vision and Language Task via Cross-modal Pre-training on Plot Table Pairs](https://mlsanity.com/paper/2305.18641/) (2023)
- [DOMINO: A Dual-System for Multi-step Visual Language Reasoning](https://mlsanity.com/paper/2310.02804/) (2023)
- [Classification-Regression for Chart Comprehension](https://mlsanity.com/paper/2111.14792/) (2021)
- [Time Series Language Model for Descriptive Caption Generation](https://mlsanity.com/paper/2501.01832/) (2025)
- [mChartQA: A universal benchmark for multimodal Chart Question Answer based on Vision-Language Alignment and Reasoning](https://mlsanity.com/paper/2404.01548/) (2024)
- [FigCaps-HF: A Figure-to-Caption Generative Framework and Benchmark with Human Feedback](https://mlsanity.com/paper/2307.10867/) (2023)
- [VProChart: Answering Chart Question through Visual Perception Alignment Agent and Programmatic Solution Reasoning](https://mlsanity.com/paper/2409.01667/) (2024)
- [Revisiting Multi-Modal LLM Evaluation](https://mlsanity.com/paper/2408.05334/) (2024)
- [DCQA: Document-Level Chart Question Answering towards Complex Reasoning and Common-Sense Understanding](https://mlsanity.com/paper/2310.18983/) (2023)
- [BloomVQA: Assessing Hierarchical Multi-modal Comprehension](https://mlsanity.com/paper/2312.12716/) (2023)
- [ChartLens: Fine-grained Visual Attribution in Charts](https://mlsanity.com/paper/2505.19360/) (2025)
- [CHARTER: heatmap-based multi-type chart data extraction](https://mlsanity.com/paper/2111.14103/) (2021)
- [GoT-CQA: Graph-of-Thought Guided Compositional Reasoning for Chart Question Answering](https://mlsanity.com/paper/2409.02611/) (2024)
- [ChartKG: A Knowledge-Graph-Based Representation for Chart Images](https://mlsanity.com/paper/2410.09761/) (2024)
- [Multimodal Information Fusion for Chart Understanding: A Survey of MLLMs -- Evolution, Limitations, and Cognitive Enhancement](https://mlsanity.com/paper/2602.10138/) (2026)
- [Q4EDA: A Novel Strategy for Textual Information Retrieval Based on User Interactions with Visual Representations of Time Series](https://mlsanity.com/paper/2101.08655/) (2021)

## Abstract (arXiv)

Chart question answering (CQA) is a newly proposed visual question answering (VQA) task where an algorithm must answer questions about data visualizations, e.g. bar charts, pie charts, and line graphs. CQA requires capabilities that natural-image VQA algorithms lack: fine-grained measurements, optical character recognition, and handling out-of-vocabulary words in both questions and answers. Without modifications, state-of-the-art VQA algorithms perform poorly on this task. Here, we propose a novel CQA algorithm called parallel recurrent fusion of image and language (PReFIL). PReFIL first learns bimodal embeddings by fusing question and image features and then intelligently aggregates these learned embeddings to answer the given question. Despite its simplicity, PReFIL greatly surpasses state-of-the art systems and human baselines on both the FigureQA and DVQA datasets. Additionally, we demonstrate that PReFIL can be used to reconstruct tables by asking a series of questions about a chart.