# Noise-Aware Training of Layout-Aware Language Models

arXiv: https://arxiv.org/abs/2404.00488 · HTML: https://mlsanity.com/paper/2404.00488/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 36 references that are in the map.
- First posted on arXiv 30 Mar 2024.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://mlsanity.com/paper/2012.14740/) (2020)
- [FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents](https://mlsanity.com/paper/1905.13538/) (2019)
- [DocFormer: End-to-End Transformer for Document Understanding](https://mlsanity.com/paper/2106.11539/) (2021)
- [SelfDoc: Self-Supervised Document Representation Learning](https://mlsanity.com/paper/2106.03331/) (2021)
- [LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding](https://mlsanity.com/paper/2202.13669/) (2022)
- [BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents](https://mlsanity.com/paper/2108.04539/) (2021)
- [Chargrid: Towards Understanding 2D Documents](https://mlsanity.com/paper/1809.08799/) (2018)
- [Self-Supervised Generalisation with Meta Auxiliary Learning](https://mlsanity.com/paper/1901.08933/) (2019)
- [Graph Convolution for Multimodal Information Extraction from Visually Rich Documents](https://mlsanity.com/paper/1903.11279/) (2019)
- [Unified Pretraining Framework for Document Understanding](https://mlsanity.com/paper/2204.10939/) (2022)
- [Semi-Supervised Sequence Modeling with Cross-View Training](https://mlsanity.com/paper/1809.08370/) (2018)
- [Spatial Dependency Parsing for Semi-Structured Document Information Extraction](https://mlsanity.com/paper/2005.00642/) (2020)
- [FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction](https://mlsanity.com/paper/2203.08411/) (2022)
- [Strong Baselines for Neural Semi-supervised Learning under Domain Shift](https://mlsanity.com/paper/1804.09530/) (2018)
- [Deep Visual Template-Free Form Parsing](https://mlsanity.com/paper/1909.02576/) (2019)
- [One-shot Text Field Labeling using Attention and Belief Propagation for Structure Information Extraction](https://mlsanity.com/paper/2009.04153/) (2020)
- [Cost-effective End-to-end Information Extraction for Semi-structured Document Images](https://mlsanity.com/paper/2104.08041/) (2021)
- [Fonduer: Knowledge Base Construction from Richly Formatted Data](https://mlsanity.com/paper/1703.05028/) (2017)
- [Tag, Copy or Predict: A Unified Weakly-Supervised Learning Framework for Visual Information Extraction using Sequences](https://mlsanity.com/paper/2106.10681/) (2021)
- [One-shot Key Information Extraction from Document with Deep Partial Graph Matching](https://mlsanity.com/paper/2109.13967/) (2021)
- [Cross-Modal Entity Matching for Visually Rich Documents](https://mlsanity.com/paper/2303.00720/) (2023)

## Abstract (arXiv)

A visually rich document (VRD) utilizes visual features along with linguistic cues to disseminate information. Training a custom extractor that identifies named entities from a document requires a large number of instances of the target document type annotated at textual and visual modalities. This is an expensive bottleneck in enterprise scenarios, where we want to train custom extractors for thousands of different document types in a scalable way. Pre-training an extractor model on unlabeled instances of the target document type, followed by a fine-tuning step on human-labeled instances does not work in these scenarios, as it surpasses the maximum allowable training time allocated for the extractor. We address this scenario by proposing a Noise-Aware Training method or NAT in this paper. Instead of acquiring expensive human-labeled documents, NAT utilizes weakly labeled documents to train an extractor in a scalable way. To avoid degradation in the model's quality due to noisy, weakly labeled samples, NAT estimates the confidence of each training sample and incorporates it as uncertainty measure during training. We train multiple state-of-the-art extractor models using NAT. Experiments on a number of publicly available and in-house datasets show that NAT-trained models are not only robust in performance -- it outperforms a transfer-learning baseline by up to 6% in terms of macro-F1 score, but it is also more label-efficient -- it reduces the amount of human-effort required to obtain comparable performance by up to 73%.