# Form-NLU: Dataset for the Form Natural Language Understanding

arXiv: https://arxiv.org/abs/2304.01577 · HTML: https://mlsanity.com/paper/2304.01577/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Apr 2023.

## Built on by (most cited first, 14 of 14)

- [A Survey of Large Language Models in Finance (FinLLMs)](https://mlsanity.com/paper/2402.02315/) (2024)
- [Deep Learning based Visually Rich Document Content Understanding: A Survey](https://mlsanity.com/paper/2408.01287/) (2024)
- [StrucTexTv3: An Efficient Vision-Language Model for Text-rich Image Perception, Comprehension, and Beyond](https://mlsanity.com/paper/2405.21013/) (2024)
- [PDF-MVQA: A Dataset for Multimodal Information Retrieval in PDF-based Visual Question Answering](https://mlsanity.com/paper/2404.12720/) (2024)
- [KVP10k : A Comprehensive Dataset for Key-Value Pair Extraction in Business Documents](https://mlsanity.com/paper/2405.00505/) (2024)
- [SynJAC: Synthetic-data-driven Joint-granular Adaptation and Calibration for Domain Specific Scanned Document Key Information Extraction](https://mlsanity.com/paper/2410.01609/) (2024)
- [BRIDGE: Benchmark for multi-hop Reasoning In long multimodal Documents with Grounded Evidence](https://mlsanity.com/paper/2603.07931/) (2026)
- [VRD-IU: Lessons from Visually Rich Document Intelligence and Understanding](https://mlsanity.com/paper/2506.01388/) (2025)
- [AiBAT: Artificial Intelligence/Instructions for Build, Assembly, and Test](https://mlsanity.com/paper/2410.02955/) (2024)
- [Workshop on Document Intelligence Understanding](https://mlsanity.com/paper/2307.16369/) (2023)
- [SynDoc: A Hybrid Discriminative-Generative Framework for Enhancing Synthetic Domain-Adaptive Document Key Information Extraction](https://mlsanity.com/paper/2509.23273/) (2025)
- [BaFCo: A Document Understanding Benchmark for Complex Bangla Form Comprehension](https://mlsanity.com/paper/2607.05614/) (2026)
- [Stop Thinking, Start Looking: Efficient Post-Training for Multimodal Document Question Answering via Reasoning-Free Alignment](https://mlsanity.com/paper/2607.14682/) (2026)
- [Mind the Gaps: A Curated Benchmark for Form Field Detection](https://mlsanity.com/paper/2609.23679/) (2026)

## Abstract (arXiv)

Compared to general document analysis tasks, form document structure understanding and retrieval are challenging. Form documents are typically made by two types of authors; A form designer, who develops the form structure and keys, and a form user, who fills out form values based on the provided keys. Hence, the form values may not be aligned with the form designer's intention (structure and keys) if a form user gets confused. In this paper, we introduce Form-NLU, the first novel dataset for form structure understanding and its key and value information extraction, interpreting the form designer's intent and the alignment of user-written value on it. It consists of 857 form images, 6k form keys and values, and 4k table keys and values. Our dataset also includes three form types: digital, printed, and handwritten, which cover diverse form appearances and layouts. We propose a robust positional and logical relation-based form key-value information extraction framework. Using this dataset, Form-NLU, we first examine strong object detection models for the form layout understanding, then evaluate the key information extraction task on the dataset, providing fine-grained results for different types of forms and keys. Furthermore, we examine it with the off-the-shelf pdf layout extraction tool and prove its feasibility in real-world cases.