# Reverse-engineering Bar Charts Using Neural Networks

arXiv: https://arxiv.org/abs/2009.02491 · HTML: https://mlsanity.com/paper/2009.02491/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Sep 2020.

## Built on by (most cited first, 8 of 8)

- [ChartSumm: A Comprehensive Benchmark for Automatic Chart Summarization of Long and Short Summaries](https://mlsanity.com/paper/2304.13620/) (2023)
- [CHARTER: heatmap-based multi-type chart data extraction](https://mlsanity.com/paper/2111.14103/) (2021)
- [Toward Automatic Interpretation of 3D Plots](https://mlsanity.com/paper/2106.07627/) (2021)
- [An extensible point-based method for data chart value detection](https://mlsanity.com/paper/2308.11788/) (2023)
- [Evaluating Graphical Perception Capabilities of Vision Transformers](https://mlsanity.com/paper/2602.18178/) (2026)
- [OsmLocator: locating overlapping scatter marks with a non-training generative perspective](https://mlsanity.com/paper/2312.11146/) (2023)
- [PaCoNet: Deep Data Extraction for Parallel Coordinates](https://mlsanity.com/paper/2608.06030/) (2026)
- [Bar-JEPA: Extracting Values from Bar Chart with Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2608.06062/) (2026)

## Abstract (arXiv)

Reverse-engineering bar charts extracts textual and numeric information from the visual representations of bar charts to support application scenarios that require the underlying information. In this paper, we propose a neural network-based method for reverse-engineering bar charts. We adopt a neural network-based object detection model to simultaneously localize and classify textual information. This approach improves the efficiency of textual information extraction. We design an encoder-decoder framework that integrates convolutional and recurrent neural networks to extract numeric information. We further introduce an attention mechanism into the framework to achieve high accuracy and robustness. Synthetic and real-world datasets are used to evaluate the effectiveness of the method. To the best of our knowledge, this work takes the lead in constructing a complete neural network-based method of reverse-engineering bar charts.