# Towards an efficient framework for Data Extraction from Chart Images

arXiv: https://arxiv.org/abs/2105.02039 · HTML: https://mlsanity.com/paper/2105.02039/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 29 references that are in the map.
- First posted on arXiv 5 May 2021.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [End-to-end people detection in crowded scenes](https://mlsanity.com/paper/1506.04878/) (2015)
- [Data Extraction from Charts via Single Deep Neural Network](https://mlsanity.com/paper/1906.11906/) (2019)
- [Chart-Text: A Fully Automated Chart Image Descriptor](https://mlsanity.com/paper/1812.10636/) (2018)
- [Scatteract: Automated extraction of data from scatter plots](https://mlsanity.com/paper/1704.06687/) (2017)

## Built on by (most cited first, 10 of 10)

- [Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction](https://mlsanity.com/paper/2410.21169/) (2024)
- [Foundation Models Meet Visualizations: Challenges and Opportunities](https://mlsanity.com/paper/2310.05771/) (2023)
- [Context-Aware Chart Element Detection](https://mlsanity.com/paper/2305.04151/) (2023)
- [LineFormer: Rethinking Line Chart Data Extraction as Instance Segmentation](https://mlsanity.com/paper/2305.01837/) (2023)
- [DCQA: Document-Level Chart Question Answering towards Complex Reasoning and Common-Sense Understanding](https://mlsanity.com/paper/2310.18983/) (2023)
- [SpaDen : Sparse and Dense Keypoint Estimation for Real-World Chart Understanding](https://mlsanity.com/paper/2308.01971/) (2023)
- [Structure Diagram Recognition in Financial Announcements](https://mlsanity.com/paper/2304.13240/) (2023)
- [An extensible point-based method for data chart value detection](https://mlsanity.com/paper/2308.11788/) (2023)
- [CHAOS: Chart Analysis with Outlier Samples](https://mlsanity.com/paper/2505.17235/) (2025)
- [Bar-JEPA: Extracting Values from Bar Chart with Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2608.06062/) (2026)

## Abstract (arXiv)

In this paper, we fill the research gap by adopting state-of-the-art computer vision techniques for the data extraction stage in a data mining system. As shown in Fig.1, this stage contains two subtasks, namely, plot element detection and data conversion. For building a robust box detector, we comprehensively compare different deep learning-based methods and find a suitable method to detect box with high precision. For building a robust point detector, a fully convolutional network with feature fusion module is adopted, which can distinguish close points compared to traditional methods. The proposed system can effectively handle various chart data without making heuristic assumptions. For data conversion, we translate the detected element into data with semantic value. A network is proposed to measure feature similarities between legends and detected elements in the legend matching phase. Furthermore, we provide a baseline on the competition of Harvesting raw tables from Infographics. Some key factors have been found to improve the performance of each stage. Experimental results demonstrate the effectiveness of the proposed system.