# CHARTER: heatmap-based multi-type chart data extraction

arXiv: https://arxiv.org/abs/2111.14103 · HTML: https://mlsanity.com/paper/2111.14103/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 17 references that are in the map.
- First posted on arXiv 28 Nov 2021.

## Builds on (most cited first, 9 of 9)

- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [FigureQA: An Annotated Figure Dataset for Visual Reasoning](https://mlsanity.com/paper/1710.07300/) (2017)
- [Character Region Awareness for Text Detection](https://mlsanity.com/paper/1904.01941/) (2019)
- [What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis](https://mlsanity.com/paper/1904.01906/) (2019)
- [Answering Questions about Data Visualizations using Efficient Bimodal Fusion](https://mlsanity.com/paper/1908.01801/) (2019)
- [Data Extraction from Charts via Single Deep Neural Network](https://mlsanity.com/paper/1906.11906/) (2019)
- [Scatteract: Automated extraction of data from scatter plots](https://mlsanity.com/paper/1704.06687/) (2017)
- [Reverse-engineering Bar Charts Using Neural Networks](https://mlsanity.com/paper/2009.02491/) (2020)

## Built on by (most cited first, 2 of 2)

- [ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules](https://mlsanity.com/paper/2304.02173/) (2023)
- [ChartDETR: A Multi-shape Detection Network for Visual Chart Recognition](https://mlsanity.com/paper/2308.07743/) (2023)

## Abstract (arXiv)

The digital conversion of information stored in documents is a great source of knowledge. In contrast to the documents text, the conversion of the embedded documents graphics, such as charts and plots, has been much less explored. We present a method and a system for end-to-end conversion of document charts into machine readable tabular data format, which can be easily stored and analyzed in the digital domain. Our approach extracts and analyses charts along with their graphical elements and supporting structures such as legends, axes, titles, and captions. Our detection system is based on neural networks, trained solely on synthetic data, eliminating the limiting factor of data collection. As opposed to previous methods, which detect graphical elements using bounding-boxes, our networks feature auxiliary domain specific heatmaps prediction enabling the precise detection of pie charts, line and scatter plots which do not fit the rectangular bounding-box presumption. Qualitative and quantitative results show high robustness and precision, improving upon previous works on popular benchmarks