# DocAnnot -- Accelerating the Creation of Key Information Extraction Datasets with GenAI-Powered Auto-annotation

arXiv: https://arxiv.org/abs/2607.24745 · HTML: https://mlsanity.com/paper/2607.24745/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 28 references that are in the map.
- First posted on arXiv 8 May 2026.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [An Introduction to Convolutional Neural Networks](https://mlsanity.com/paper/1511.08458/) (2015)
- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://mlsanity.com/paper/2012.14740/) (2020)
- [FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents](https://mlsanity.com/paper/1905.13538/) (2019)
- [Unifying Vision, Text, and Layout for Universal Document Processing](https://mlsanity.com/paper/2212.02623/) (2022)
- [LLMs Accelerate Annotation for Medical Information Extraction](https://mlsanity.com/paper/2312.02296/) (2023)
- [DocFormerv2: Local Features for Document Understanding](https://mlsanity.com/paper/2306.01733/) (2023)
- [Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction](https://mlsanity.com/paper/2410.21169/) (2024)
- [Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation](https://mlsanity.com/paper/2406.12114/) (2024)
- [Business Document Information Extraction: Towards Practical Benchmarks](https://mlsanity.com/paper/2206.11229/) (2022)
- [GenKIE: Robust Generative Multimodal Document Key Information Extraction](https://mlsanity.com/paper/2310.16131/) (2023)
- [Key Information Extraction From Documents: Evaluation And Generator](https://mlsanity.com/paper/2106.14624/) (2021)
- [KVP10k : A Comprehensive Dataset for Key-Value Pair Extraction in Business Documents](https://mlsanity.com/paper/2405.00505/) (2024)
- [DocKD: Knowledge Distillation from LLMs for Open-World Document Understanding Models](https://mlsanity.com/paper/2410.03061/) (2024)

## Abstract (arXiv)

Key Information Extraction (KIE) is vital for many document applications, but creating training datasets is traditionally a time-consuming manual process. We introduce DocAnnot, a framework that significantly accelerates KIE dataset generation. DocAnnot leverages a Large Vision Language Model (LVLM) for label value extraction, OCR for text/bounding box detection, and a novel Spatially Informed Contextual Matching (SICM) algorithm. SICM improves label-value association by combining spatial relationships and proximity analysis with textual matching. We evaluate our framework on the CORD and SROIE benchmarks, demonstrating its ability to auto-generate annotations with F1-scores of 0.679 and 0.846, respectively. Furthermore, we investigate the effectiveness of using auto-annotated data for fine-tuning downstream KIE models. While human-annotated data remains superior, models trained exclusively on DocAnnot's outputs attain respectable performance (e.g., LayoutLMv3 achieving an F1-score of 0.6765 on CORD). These results show that while our framework significantly reduces reliance on manual effort, it does not yet fully eliminate the need for human intervention. However, by automating the process to a point where reviewers can efficiently refine outputs, our system enables near-perfect annotations with much greater efficiency than manual annotation from scratch. This approach offers substantial time and cost savings, making it valuable for resource-constrained settings and rapid model prototyping.