# End-to-End Information Extraction by Character-Level Embedding and Multi-Stage Attentional U-Net

arXiv: https://arxiv.org/abs/2106.00952 · HTML: https://mlsanity.com/paper/2106.00952/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 33 references that are in the map.
- First posted on arXiv 2 Jun 2021.

## Builds on (most cited first, 11 of 11)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [Chargrid: Towards Understanding 2D Documents](https://mlsanity.com/paper/1809.08799/) (2018)
- [Learning to Extract Semantic Structure from Documents Using Multimodal Fully Convolutional Neural Network](https://mlsanity.com/paper/1706.02337/) (2017)
- [dhSegment: A generic deep-learning approach for document segmentation](https://mlsanity.com/paper/1804.10371/) (2018)
- [Attend, Copy, Parse -- End-to-end information extraction from documents](https://mlsanity.com/paper/1812.07248/) (2018)
- [CU-Net: Coupled U-Nets](https://mlsanity.com/paper/1808.06521/) (2018)

## Built on by (most cited first, 4 of 4)

- [Reading Order Matters: Information Extraction from Visually-rich Documents by Token Path Prediction](https://mlsanity.com/paper/2310.11016/) (2023)
- [A Question-Answering Approach to Key Value Pair Extraction from Form-like Document Images](https://mlsanity.com/paper/2304.07957/) (2023)
- [HYCEDIS: HYbrid Confidence Engine for Deep Document Intelligence System](https://mlsanity.com/paper/2206.02628/) (2022)
- [VRDSynth: Synthesizing Programs for Multilingual Visually Rich Document Information Extraction](https://mlsanity.com/paper/2407.06826/) (2024)

## Abstract (arXiv)

Information extraction from document images has received a lot of attention recently, due to the need for digitizing a large volume of unstructured documents such as invoices, receipts, bank transfers, etc. In this paper, we propose a novel deep learning architecture for end-to-end information extraction on the 2D character-grid embedding of the document, namely the textitMulti-Stage Attentional U-Net. To effectively capture the textual and spatial relations between 2D elements, our model leverages a specialized multi-stage encoder-decoders design, in conjunction with efficient uses of the self-attention mechanism and the box convolution. Experimental results on different datasets show that our model outperforms the baseline U-Net architecture by a large margin while using 40% fewer parameters. Moreover, it also significantly improved the baseline in erroneous OCR and limited training data scenario, thus becomes practical for real-world applications.