# Deep Visual Template-Free Form Parsing

arXiv: https://arxiv.org/abs/1909.02576 · HTML: https://mlsanity.com/paper/1909.02576/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Sep 2019.

## Built on by (most cited first, 24 of 24)

- [OCR-free Document Understanding Transformer](https://mlsanity.com/paper/2111.15664/) (2021)
- [DocFormer: End-to-End Transformer for Document Understanding](https://mlsanity.com/paper/2106.11539/) (2021)
- [End-to-end Document Recognition and Understanding with Dessurt](https://mlsanity.com/paper/2203.16618/) (2022)
- [FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction](https://mlsanity.com/paper/2203.08411/) (2022)
- [DocILE Benchmark for Document Information Localization and Extraction](https://mlsanity.com/paper/2302.05658/) (2023)
- [SlowFast-LLaVA-1.5: A Family of Token-Efficient Video Large Language Models for Long-Form Video Understanding](https://mlsanity.com/paper/2503.18943/) (2025)
- [PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Models](https://mlsanity.com/paper/2412.09613/) (2024)
- [ROPE: Reading Order Equivariant Positional Encoding for Graph-based Document Information Extraction](https://mlsanity.com/paper/2106.10786/) (2021)
- [Doc2Graph: a Task Agnostic Document Understanding Framework based on Graph Neural Networks](https://mlsanity.com/paper/2208.11168/) (2022)
- [Post-OCR Paragraph Recognition by Graph Convolutional Networks](https://mlsanity.com/paper/2101.12741/) (2021)
- [Visual FUDGE: Form Understanding via Dynamic Graph Editing](https://mlsanity.com/paper/2105.08194/) (2021)
- [HoVLE: Unleashing the Power of Monolithic Vision-Language Models with Holistic Vision-Language Embedding](https://mlsanity.com/paper/2412.16158/) (2024)
- [A Question-Answering Approach to Key Value Pair Extraction from Form-like Document Images](https://mlsanity.com/paper/2304.07957/) (2023)
- [BuDDIE: A Business Document Dataset for Multi-task Information Extraction](https://mlsanity.com/paper/2404.04003/) (2024)
- [Transformers and Language Models in Form Understanding: A Comprehensive Review of Scanned Document Analysis](https://mlsanity.com/paper/2403.04080/) (2024)
- [Let's Fuse Step by Step: A Generative Fusion Decoding Algorithm with LLMs for Robust and Instruction-Aware ASR and OCR](https://mlsanity.com/paper/2405.14259/) (2024)
- [CoMemo: LVLMs Need Image Context with Image Memory](https://mlsanity.com/paper/2506.06279/) (2025)
- [CommonForms: A Large, Diverse Dataset for Form Field Detection](https://mlsanity.com/paper/2509.16506/) (2025)
- [TreeForm: End-to-end Annotation and Evaluation for Form Document Parsing](https://mlsanity.com/paper/2402.05282/) (2024)
- [GeoContrastNet: Contrastive Key-Value Edge Learning for Language-Agnostic Document Understanding](https://mlsanity.com/paper/2405.03104/) (2024)
- [Language Independent Neuro-Symbolic Semantic Parsing for Form Understanding](https://mlsanity.com/paper/2305.04460/) (2023)
- [Flowmind2Digital: The First Comprehensive Flowmind Recognition and Conversion Approach](https://mlsanity.com/paper/2401.03742/) (2024)
- [DELINE8K: A Synthetic Data Pipeline for the Semantic Segmentation of Historical Documents](https://mlsanity.com/paper/2404.19259/) (2024)
- [Noise-Aware Training of Layout-Aware Language Models](https://mlsanity.com/paper/2404.00488/) (2024)

## Abstract (arXiv)

Automatic, template-free extraction of information from form images is challenging due to the variety of form layouts. This is even more challenging for historical forms due to noise and degradation. A crucial part of the extraction process is associating input text with pre-printed labels. We present a learned, template-free solution to detecting pre-printed text and input text/handwriting and predicting pair-wise relationships between them. While previous approaches to this problem have been focused on clean images and clear layouts, we show our approach is effective in the domain of noisy, degraded, and varied form images. We introduce a new dataset of historical form images (late 1800s, early 1900s) for training and validating our approach. Our method uses a convolutional network to detect pre-printed text and input text lines. We pool features from the detection network to classify possible relationships in a language-agnostic way. We show that our proposed pairing method outperforms heuristic rules and that visual features are critical to obtaining high accuracy.