# Document-level Entity-based Extraction as Template Generation

arXiv: https://arxiv.org/abs/2109.04901 · HTML: https://mlsanity.com/paper/2109.04901/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 10 Sep 2021.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned](https://mlsanity.com/paper/1905.09418/) (2019)
- [Are Sixteen Heads Really Better than One?](https://mlsanity.com/paper/1905.10650/) (2019)
- [Entity, Relation, and Event Extraction with Contextualized Span Representations](https://mlsanity.com/paper/1909.03546/) (2019)
- [Structured Prediction as Translation between Augmented Natural Languages](https://mlsanity.com/paper/2101.05779/) (2021)
- [Document-Level Event Argument Extraction by Conditional Generation](https://mlsanity.com/paper/2104.05919/) (2021)
- [SciREX: A Challenge Dataset for Document-Level Information Extraction](https://mlsanity.com/paper/2005.00512/) (2020)
- [Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs](https://mlsanity.com/paper/1909.00228/) (2019)
- [Document-Level N-ary Relation Extraction with Multiscale Representation Learning](https://mlsanity.com/paper/1904.02347/) (2019)
- [Contrastive Triple Extraction with Generative Transformer](https://mlsanity.com/paper/2009.06207/) (2020)
- [Document-Level Event Role Filler Extraction using Multi-Granularity Contextualized Encoding](https://mlsanity.com/paper/2005.06579/) (2020)
- [Document-level Event Extraction with Efficient End-to-end Learning of Cross-event Dependencies](https://mlsanity.com/paper/2010.12787/) (2020)
- [Understanding Clinical Trial Reports: Extracting Medical Entities and Their Relations](https://mlsanity.com/paper/2010.03550/) (2020)
- [Extracting Summary Knowledge Graphs from Long Documents](https://mlsanity.com/paper/2009.09162/) (2020)

## Built on by (most cited first, 30 of 33)

- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [Code4Struct: Code Generation for Few-Shot Event Structure Prediction](https://mlsanity.com/paper/2210.12810/) (2022)
- [Generative Knowledge Graph Construction: A Review](https://mlsanity.com/paper/2210.12714/) (2022)
- [Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction](https://mlsanity.com/paper/2203.08308/) (2022)
- [Summarization as Indirect Supervision for Relation Extraction](https://mlsanity.com/paper/2205.09837/) (2022)
- [DICE: Data-Efficient Clinical Event Extraction with Generative Models](https://mlsanity.com/paper/2208.07989/) (2022)
- [AMPERE: AMR-Aware Prefix for Generation-Based Event Argument Extraction Model](https://mlsanity.com/paper/2305.16734/) (2023)
- [Retrieval-Augmented Generative Question Answering for Event Argument Extraction](https://mlsanity.com/paper/2211.07067/) (2022)
- [Generative Prompt Tuning for Relation Classification](https://mlsanity.com/paper/2210.12435/) (2022)
- [Dynamic Global Memory for Document-level Argument Extraction](https://mlsanity.com/paper/2209.08679/) (2022)
- [Iterative Document-level Information Extraction via Imitation Learning](https://mlsanity.com/paper/2210.06600/) (2022)
- [A Unified View of Evaluation Metrics for Structured Prediction](https://mlsanity.com/paper/2310.13793/) (2023)
- [ReSel: N-ary Relation Extraction from Scientific Text and Tables by Learning to Retrieve and Select](https://mlsanity.com/paper/2210.14427/) (2022)
- [On the Evolution of Knowledge Graphs: A Survey and Perspective](https://mlsanity.com/paper/2310.04835/) (2023)
- [Controllable Text Generation for Open-Domain Creativity and Fairness](https://mlsanity.com/paper/2209.12099/) (2022)
- [Schema-aware Reference as Prompt Improves Data-Efficient Knowledge Graph Construction](https://mlsanity.com/paper/2210.10709/) (2022)
- [FactReranker: Fact-guided Reranker for Faithful Radiology Report Summarization](https://mlsanity.com/paper/2303.08335/) (2023)
- [Learning Action Conditions from Instructional Manuals for Instruction Understanding](https://mlsanity.com/paper/2205.12420/) (2022)
- [Document-Level Multi-Event Extraction with Event Proxy Nodes and Hausdorff Distance Minimization](https://mlsanity.com/paper/2305.18926/) (2023)
- [DORE: Document Ordered Relation Extraction based on Generative Framework](https://mlsanity.com/paper/2210.16064/) (2022)
- [Harnessing Knowledge and Reasoning for Human-Like Natural Language Generation: A Brief Review](https://mlsanity.com/paper/2212.03747/) (2022)
- [MultiMUC: Multilingual Template Filling on MUC-4](https://mlsanity.com/paper/2401.16209/) (2024)
- [On Event Individuation for Document-Level Information Extraction](https://mlsanity.com/paper/2212.09702/) (2022)
- [ManiTweet: A New Benchmark for Identifying Manipulation of News on Social Media](https://mlsanity.com/paper/2305.14225/) (2023)
- [FAMuS: Frames Across Multiple Sources](https://mlsanity.com/paper/2311.05601/) (2023)
- [A Structure-aware Generative Model for Biomedical Event Extraction](https://mlsanity.com/paper/2408.06583/) (2024)
- [Scalable Detection of Salient Entities in News Articles](https://mlsanity.com/paper/2405.20461/) (2024)
- [SciEvent: Benchmarking Multi-domain Scientific Event Extraction](https://mlsanity.com/paper/2509.15620/) (2025)
- [Argument Mining as a Text-to-Text Generation Task](https://mlsanity.com/paper/2603.23949/) (2026)
- [DLUE: Benchmarking Document Language Understanding](https://mlsanity.com/paper/2305.09520/) (2023)

## Abstract (arXiv)

Document-level entity-based extraction (EE), aiming at extracting entity-centric information such as entity roles and entity relations, is key to automatic knowledge acquisition from text corpora for various domains. Most document-level EE systems build extractive models, which struggle to model long-term dependencies among entities at the document level. To address this issue, we propose a generative framework for two document-level EE tasks: role-filler entity extraction (REE) and relation extraction (RE). We first formulate them as a template generation problem, allowing models to efficiently capture cross-entity dependencies, exploit label semantics, and avoid the exponential computation complexity of identifying N-ary relations. A novel cross-attention guided copy mechanism, TopK Copy, is incorporated into a pre-trained sequence-to-sequence model to enhance the capabilities of identifying key information in the input document. Experiments done on the MUC-4 and SciREX dataset show new state-of-the-art results on REE (+3.26%), binary RE (+4.8%), and 4-ary RE (+2.7%) in F1 score.