# A sequence-to-sequence approach for document-level relation extraction

arXiv: https://arxiv.org/abs/2204.01098 · HTML: https://mlsanity.com/paper/2204.01098/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 81 references that are in the map.
- First posted on arXiv 3 Apr 2022.

## Builds on (most cited first, 30 of 41)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Long Range Arena: A Benchmark for Efficient Transformers](https://mlsanity.com/paper/2011.04006/) (2020)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction](https://mlsanity.com/paper/1808.09602/) (2018)
- [Entity, Relation, and Event Extraction with Contextualized Span Representations](https://mlsanity.com/paper/1909.03546/) (2019)
- [DocRED: A Large-Scale Document-Level Relation Extraction Dataset](https://mlsanity.com/paper/1906.06127/) (2019)
- [Revisiting Few-sample BERT Fine-tuning](https://mlsanity.com/paper/2006.05987/) (2020)
- [Order Matters: Sequence to sequence for sets](https://mlsanity.com/paper/1511.06391/) (2015)
- [Incorporating Copying Mechanism in Sequence-to-Sequence Learning](https://mlsanity.com/paper/1603.06393/) (2016)
- [End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures](https://mlsanity.com/paper/1601.00770/) (2016)
- [Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling](https://mlsanity.com/paper/2010.11304/) (2020)
- [SciREX: A Challenge Dataset for Document-Level Information Extraction](https://mlsanity.com/paper/2005.00512/) (2020)
- [Cross-Sentence N-ary Relation Extraction with Graph LSTMs](https://mlsanity.com/paper/1708.03743/) (2017)
- [SciFive: a text-to-text transformer model for biomedical literature](https://mlsanity.com/paper/2106.03598/) (2021)
- [Reasoning with Latent Structure Refinement for Document-Level Relation Extraction](https://mlsanity.com/paper/2005.06312/) (2020)
- [An Improved Baseline for Sentence-level Relation Extraction](https://mlsanity.com/paper/2102.01373/) (2021)
- [Effective Modeling of Encoder-Decoder Architecture for Joint Entity and Relation Extraction](https://mlsanity.com/paper/1911.09886/) (2019)
- [Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs](https://mlsanity.com/paper/1909.00228/) (2019)
- [Entity Structure Within and Throughout: Modeling Mention Dependencies for Document-Level Relation Extraction](https://mlsanity.com/paper/2102.10249/) (2021)
- [Joint entity recognition and relation extraction as a multi-head selection problem](https://mlsanity.com/paper/1804.07847/) (2018)
- [Joint Entity and Relation Extraction with Set Prediction Networks](https://mlsanity.com/paper/2011.01675/) (2020)
- [Document-Level N-ary Relation Extraction with Multiscale Representation Learning](https://mlsanity.com/paper/1904.02347/) (2019)
- [CopyMTL: Copy Mechanism for Joint Extraction of Entities and Relations with Multi-Task Learning](https://mlsanity.com/paper/1911.10438/) (2019)
- [Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction](https://mlsanity.com/paper/1802.10569/) (2018)

## Built on by (most cited first, 25 of 25)

- [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://mlsanity.com/paper/2210.10341/) (2022)
- [Structured information extraction from complex scientific text with fine-tuned large language models](https://mlsanity.com/paper/2212.05238/) (2022)
- [Generative Knowledge Graph Construction: A Review](https://mlsanity.com/paper/2210.12714/) (2022)
- [Consistency Guided Knowledge Retrieval and Denoising in LLMs for Zero-shot Document-level Relation Triplet Extraction](https://mlsanity.com/paper/2401.13598/) (2024)
- [Beyond Rule-based Named Entity Recognition and Relation Extraction for Process Model Generation from Natural Language Text](https://mlsanity.com/paper/2305.03960/) (2023)
- [FactReranker: Fact-guided Reranker for Faithful Radiology Report Summarization](https://mlsanity.com/paper/2303.08335/) (2023)
- [An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction](https://mlsanity.com/paper/2401.01326/) (2024)
- [Rethinking Document-Level Relation Extraction: A Reality Check](https://mlsanity.com/paper/2306.08953/) (2023)
- [Empirical Analysis of Dialogue Relation Extraction with Large Language Models](https://mlsanity.com/paper/2404.17802/) (2024)
- [DORE: Document Ordered Relation Extraction based on Generative Framework](https://mlsanity.com/paper/2210.16064/) (2022)
- [End-to-End n-ary Relation Extraction for Combination Drug Therapies](https://mlsanity.com/paper/2303.16886/) (2023)
- [Document-Level In-Context Few-Shot Relation Extraction via Pre-Trained Language Models](https://mlsanity.com/paper/2310.11085/) (2023)
- [Comparison of pipeline, sequence-to-sequence, and GPT models for end-to-end relation extraction: experiments with the rare disease use-case](https://mlsanity.com/paper/2311.13729/) (2023)
- [A Benchmark for End-to-End Zero-Shot Biomedical Relation Extraction with LLMs: Experiments with OpenAI Models](https://mlsanity.com/paper/2504.04083/) (2025)
- [Relation Extraction in underexplored biomedical domains: A diversity-optimised sampling and synthetic data generation approach](https://mlsanity.com/paper/2311.06364/) (2023)
- [Chem-FINESE: Validating Fine-Grained Few-shot Entity Extraction through Text Reconstruction](https://mlsanity.com/paper/2401.10189/) (2024)
- [How Important is Domain Specificity in Language Models and Instruction Finetuning for Biomedical Relation Extraction?](https://mlsanity.com/paper/2402.13470/) (2024)
- [Extracting Polymer Nanocomposite Samples from Full-Length Documents](https://mlsanity.com/paper/2403.00260/) (2024)
- [Automated Text Mining of Experimental Methodologies from Biomedical Literature](https://mlsanity.com/paper/2404.13779/) (2024)
- [Predicting drug-gene relations via analogy tasks with word embeddings](https://mlsanity.com/paper/2406.00984/) (2024)
- [On the Robustness of Document-Level Relation Extraction Models to Entity Name Variations](https://mlsanity.com/paper/2406.07444/) (2024)
- [Question-Answer Extraction from Scientific Articles Using Knowledge Graphs and Large Language Models](https://mlsanity.com/paper/2507.13827/) (2025)
- [ComProScanner: A multi-agent based framework for composition-property structured data extraction from scientific literature](https://mlsanity.com/paper/2510.20362/) (2025)
- [Joint Learning-based Causal Relation Extraction from Biomedical Literature](https://mlsanity.com/paper/2208.01316/) (2022)
- [Similarity-based Memory Enhanced Joint Entity and Relation Extraction](https://mlsanity.com/paper/2307.11762/) (2023)

## Abstract (arXiv)

Motivated by the fact that many relations cross the sentence boundary, there has been increasing interest in document-level relation extraction (DocRE). DocRE requires integrating information within and across sentences, capturing complex interactions between mentions of entities. Most existing methods are pipeline-based, requiring entities as input. However, jointly learning to extract entities and relations can improve performance and be more efficient due to shared parameters and training steps. In this paper, we develop a sequence-to-sequence approach, seq2rel, that can learn the subtasks of DocRE (entity extraction, coreference resolution and relation extraction) end-to-end, replacing a pipeline of task-specific components. Using a simple strategy we call entity hinting, we compare our approach to existing pipeline-based methods on several popular biomedical datasets, in some cases exceeding their performance. We also report the first end-to-end results on these datasets for future comparison. Finally, we demonstrate that, under our model, an end-to-end approach outperforms a pipeline-based approach. Our code, data and trained models are available at urlhttps://github.com/johngiorgi/seq2rel. An online demo is available at urlhttps://share.streamlit.io/johngiorgi/seq2rel/main/demo.py.