# One-shot Text Field Labeling using Attention and Belief Propagation for Structure Information Extraction

arXiv: https://arxiv.org/abs/2009.04153 · HTML: https://mlsanity.com/paper/2009.04153/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2020.

## Built on by (most cited first, 16 of 16)

- [StrucTexT: Structured Text Understanding with Multi-Modal Transformers](https://mlsanity.com/paper/2108.02923/) (2021)
- [ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding](https://mlsanity.com/paper/2210.06155/) (2022)
- [Document AI: Benchmarks, Models and Applications](https://mlsanity.com/paper/2111.08609/) (2021)
- [WebFormer: The Web-page Transformer for Structure Information Extraction](https://mlsanity.com/paper/2202.00217/) (2022)
- [Deep Learning based Visually Rich Document Content Understanding: A Survey](https://mlsanity.com/paper/2408.01287/) (2024)
- [MatchVIE: Exploiting Match Relevancy between Entities for Visual Information Extraction](https://mlsanity.com/paper/2106.12940/) (2021)
- [Enhancing Visually-Rich Document Understanding via Layout Structure Modeling](https://mlsanity.com/paper/2308.07777/) (2023)
- [Deep Learning based Key Information Extraction from Business Documents: Systematic Literature Review](https://mlsanity.com/paper/2408.06345/) (2024)
- [Doc2SoarGraph: Discrete Reasoning over Visually-Rich Table-Text Documents via Semantic-Oriented Hierarchical Graphs](https://mlsanity.com/paper/2305.01938/) (2023)
- [PPN: Parallel Pointer-based Network for Key Information Extraction with Complex Layouts](https://mlsanity.com/paper/2307.10551/) (2023)
- [One-shot Key Information Extraction from Document with Deep Partial Graph Matching](https://mlsanity.com/paper/2109.13967/) (2021)
- [Knowing Where and What: Unified Word Block Pretraining for Document Understanding](https://mlsanity.com/paper/2207.13979/) (2022)
- [GVdoc: Graph-based Visual Document Classification](https://mlsanity.com/paper/2305.17219/) (2023)
- [Towards Human-Like Machine Comprehension: Few-Shot Relational Learning in Visually-Rich Documents](https://mlsanity.com/paper/2403.15765/) (2024)
- [Noise-Aware Training of Layout-Aware Language Models](https://mlsanity.com/paper/2404.00488/) (2024)
- [Visual Information Extraction from Documents via Classification-Guided Large Vision-Language Models](https://mlsanity.com/paper/2607.22723/) (2026)

## Abstract (arXiv)

Structured information extraction from document images usually consists of three steps: text detection, text recognition, and text field labeling. While text detection and text recognition have been heavily studied and improved a lot in literature, text field labeling is less explored and still faces many challenges. Existing learning based methods for text labeling task usually require a large amount of labeled examples to train a specific model for each type of document. However, collecting large amounts of document images and labeling them is difficult and sometimes impossible due to privacy issues. Deploying separate models for each type of document also consumes a lot of resources. Facing these challenges, we explore one-shot learning for the text field labeling task. Existing one-shot learning methods for the task are mostly rule-based and have difficulty in labeling fields in crowded regions with few landmarks and fields consisting of multiple separate text regions. To alleviate these problems, we proposed a novel deep end-to-end trainable approach for one-shot text field labeling, which makes use of attention mechanism to transfer the layout information between document images. We further applied conditional random field on the transferred layout information for the refinement of field labeling. We collected and annotated a real-world one-shot field labeling dataset with a large variety of document types and conducted extensive experiments to examine the effectiveness of the proposed model. To stimulate research in this direction, the collected dataset and the one-shot model will be released1.