# Doc2Dict: Information Extraction as Text Generation

arXiv: https://arxiv.org/abs/2105.07510 · HTML: https://mlsanity.com/paper/2105.07510/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 16 May 2021.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [Training Deep Nets with Sublinear Memory Cost](https://mlsanity.com/paper/1604.06174/) (2016)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [The Natural Language Decathlon: Multitask Learning as Question Answering](https://mlsanity.com/paper/1806.08730/) (2018)
- [Named Entity Recognition with Bidirectional LSTM-CNNs](https://mlsanity.com/paper/1511.08308/) (2015)
- [Structured Prediction as Translation between Augmented Natural Languages](https://mlsanity.com/paper/2101.05779/) (2021)
- [MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2009.12005/) (2020)
- [Effective Modeling of Encoder-Decoder Architecture for Joint Entity and Relation Extraction](https://mlsanity.com/paper/1911.09886/) (2019)
- [SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching](https://mlsanity.com/paper/2005.05298/) (2020)
- [Kleister: A novel task for Information Extraction involving Long Documents with Complex Layout](https://mlsanity.com/paper/2003.02356/) (2020)
- [Attend, Copy, Parse -- End-to-end information extraction from documents](https://mlsanity.com/paper/1812.07248/) (2018)
- [Robust Lexical Features for Improved Neural Network Named-Entity Recognition](https://mlsanity.com/paper/1806.03489/) (2018)
- [Rapid Adaptation of BERT for Information Extraction on Domain-Specific Business Documents](https://mlsanity.com/paper/2002.01861/) (2020)
- [SeqGenSQL -- A Robust Sequence Generation Model for Structured Query Language](https://mlsanity.com/paper/2011.03836/) (2020)

## Built on by (most cited first, 6 of 6)

- [Structured information extraction from complex scientific text with fine-tuned large language models](https://mlsanity.com/paper/2212.05238/) (2022)
- [STable: Table Generation Framework for Encoder-Decoder Models](https://mlsanity.com/paper/2206.04045/) (2022)
- [RealKIE: Five Novel Datasets for Enterprise Key Information Extraction](https://mlsanity.com/paper/2403.20101/) (2024)
- [End-to-End Document Classification and Key Information Extraction using Assignment Optimization](https://mlsanity.com/paper/2306.00750/) (2023)
- [A Semantic Parsing Framework for End-to-End Time Normalization](https://mlsanity.com/paper/2507.06450/) (2025)
- [ComProScanner: A multi-agent based framework for composition-property structured data extraction from scientific literature](https://mlsanity.com/paper/2510.20362/) (2025)

## Abstract (arXiv)

Typically, information extraction (IE) requires a pipeline approach: first, a sequence labeling model is trained on manually annotated documents to extract relevant spans; then, when a new document arrives, a model predicts spans which are then post-processed and standardized to convert the information into a database entry. We replace this labor-intensive workflow with a transformer language model trained on existing database records to directly generate structured JSON. Our solution removes the workload associated with producing token-level annotations and takes advantage of a data source which is generally quite plentiful (e.g. database records). As long documents are common in information extraction tasks, we use gradient checkpointing and chunked encoding to apply our method to sequences of up to 32,000 tokens on a single GPU. Our Doc2Dict approach is competitive with more complex, hand-engineered pipelines and offers a simple but effective baseline for document-level information extraction. We release our Doc2Dict model and code to reproduce our experiments and facilitate future work.