# An extensible point-based method for data chart value detection

arXiv: https://arxiv.org/abs/2308.11788 · HTML: https://mlsanity.com/paper/2308.11788/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 46 references that are in the map.
- First posted on arXiv 22 Aug 2023.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [CORD-19: The COVID-19 Open Research Dataset](https://mlsanity.com/paper/2004.10706/) (2020)
- [DocBank: A Benchmark Dataset for Document Layout Analysis](https://mlsanity.com/paper/2006.01038/) (2020)
- [End-to-end people detection in crowded scenes](https://mlsanity.com/paper/1506.04878/) (2015)
- [dhSegment: A generic deep-learning approach for document segmentation](https://mlsanity.com/paper/1804.10371/) (2018)
- [Data Extraction from Charts via Single Deep Neural Network](https://mlsanity.com/paper/1906.11906/) (2019)
- [IIIT-AR-13K: A New Dataset for Graphical Object Detection in Documents](https://mlsanity.com/paper/2008.02569/) (2020)
- [Scatteract: Automated extraction of data from scatter plots](https://mlsanity.com/paper/1704.06687/) (2017)
- [Towards an efficient framework for Data Extraction from Chart Images](https://mlsanity.com/paper/2105.02039/) (2021)
- [Reverse-engineering Bar Charts Using Neural Networks](https://mlsanity.com/paper/2009.02491/) (2020)
- [Point Proposal Network for Reconstructing 3D Particle Endpoints with Sub-Pixel Precision in Liquid Argon Time Projection Chambers](https://mlsanity.com/paper/2006.14745/) (2020)
- [Point Proposal Network: Accelerating Point Source Detection Through Deep Learning](https://mlsanity.com/paper/2008.02093/) (2020)

## Built on by (most cited first, 1 of 1)

- [Bar-JEPA: Extracting Values from Bar Chart with Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2608.06062/) (2026)

## Abstract (arXiv)

We present an extensible method for identifying semantic points to reverse engineer (i.e. extract the values of) data charts, particularly those in scientific articles. Our method uses a point proposal network (akin to region proposal networks for object detection) to directly predict the position of points of interest in a chart, and it is readily extensible to multiple chart types and chart elements. We focus on complex bar charts in the scientific literature, on which our model is able to detect salient points with an accuracy of 0.8705 F1 (@1.5-cell max deviation); it achieves 0.9810 F1 on synthetically-generated charts similar to those used in prior works. We also explore training exclusively on synthetic data with novel augmentations, reaching surprisingly competent performance in this way (0.6621 F1) on real charts with widely varying appearance, and we further demonstrate our unchanged method applied directly to synthetic pie charts (0.8343 F1). Datasets, trained models, and evaluation code are available at https://github.com/BNLNLP/PPN_model.