# Jointly Learning Span Extraction and Sequence Labeling for Information Extraction from Business Documents

arXiv: https://arxiv.org/abs/2205.13434 · HTML: https://mlsanity.com/paper/2205.13434/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 32 references that are in the map.
- First posted on arXiv 26 May 2022.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review](https://mlsanity.com/paper/2103.06268/) (2021)
- [A Unified MRC Framework for Named Entity Recognition](https://mlsanity.com/paper/1910.11476/) (2019)
- [Automated Concatenation of Embeddings for Structured Prediction](https://mlsanity.com/paper/2010.05006/) (2020)
- [Few-Shot Question Answering by Pretraining Span Selection](https://mlsanity.com/paper/2101.00438/) (2021)
- [A Span-Based Model for Joint Overlapped and Discontinuous Named Entity Recognition](https://mlsanity.com/paper/2106.14373/) (2021)
- [TXtract: Taxonomy-Aware Knowledge Extraction for Thousands of Product Categories](https://mlsanity.com/paper/2004.13852/) (2020)
- [Empower Sequence Labeling with Task-Aware Neural Language Model](https://mlsanity.com/paper/1709.04109/) (2017)
- [Nested Named Entity Recognition with Partially-Observed TreeCRFs](https://mlsanity.com/paper/2012.08478/) (2020)
- [Rapid Adaptation of BERT for Information Extraction on Domain-Specific Business Documents](https://mlsanity.com/paper/2002.01861/) (2020)
- [A Span Extraction Approach for Information Extraction on Visually-Rich Documents](https://mlsanity.com/paper/2106.00978/) (2021)

## Built on by (most cited first, 4 of 4)

- [Scoping Review of Active Learning Strategies and their Evaluation Environments for Entity Recognition Tasks](https://mlsanity.com/paper/2407.03895/) (2024)
- [Towards Safer Operations: An Expert-involved Dataset of High-Pressure Gas Incidents for Preventing Future Failures](https://mlsanity.com/paper/2310.12074/) (2023)
- [Improving Document Image Understanding with Reinforcement Finetuning](https://mlsanity.com/paper/2209.12561/) (2022)
- [CareLab at #SMM4H-HeaRD 2025: Insomnia Detection and Food Safety Event Extraction with Domain-Aware Transformers](https://mlsanity.com/paper/2506.18185/) (2025)

## Abstract (arXiv)

This paper introduces a new information extraction model for business documents. Different from prior studies which only base on span extraction or sequence labeling, the model takes into account advantage of both span extraction and sequence labeling. The combination allows the model to deal with long documents with sparse information (the small amount of extracted information). The model is trained end-to-end to jointly optimize the two tasks in a unified manner. Experimental results on four business datasets in English and Japanese show that the model achieves promising results and is significantly faster than the normal span-based extraction method. The code is also available.