# SpaDen : Sparse and Dense Keypoint Estimation for Real-World Chart Understanding

arXiv: https://arxiv.org/abs/2308.01971 · HTML: https://mlsanity.com/paper/2308.01971/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 27 references that are in the map.
- First posted on arXiv 3 Aug 2023.

## Builds on (most cited first, 12 of 12)

- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [Deep Layer Aggregation](https://mlsanity.com/paper/1707.06484/) (2017)
- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [DocFormer: End-to-End Transformer for Document Understanding](https://mlsanity.com/paper/2106.11539/) (2021)
- [Unified Pretraining Framework for Document Understanding](https://mlsanity.com/paper/2204.10939/) (2022)
- [Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation](https://mlsanity.com/paper/1911.10529/) (2019)
- [Towards an efficient framework for Data Extraction from Chart Images](https://mlsanity.com/paper/2105.02039/) (2021)

## Built on by (most cited first, 2 of 2)

- [ChartReformer: Natural Language-Driven Chart Image Editing](https://mlsanity.com/paper/2403.00209/) (2024)
- [Back2Struct: Making Structured Images Editable Again](https://mlsanity.com/paper/2609.37016/) (2026)

## Abstract (arXiv)

We introduce a novel bottom-up approach for the extraction of chart data. Our model utilizes images of charts as inputs and learns to detect keypoints (KP), which are used to reconstruct the components within the plot area. Our novelty lies in detecting a fusion of continuous and discrete KP as predicted heatmaps. A combination of sparse and dense per-pixel objectives coupled with a uni-modal self-attention-based feature-fusion layer is applied to learn KP embeddings. Further leveraging deep metric learning for unsupervised clustering, allows us to segment the chart plot area into various objects. By further matching the chart components to the legend, we are able to obtain the data series names. A post-processing threshold is applied to the KP embeddings to refine the object reconstructions and improve accuracy. Our extensive experiments include an evaluation of different modules for KP estimation and the combination of deep layer aggregation and corner pooling approaches. The results of our experiments provide extensive evaluation for the task of real-world chart data extraction.