# DocRED: A Large-Scale Document-Level Relation Extraction Dataset

arXiv: https://arxiv.org/abs/1906.06127 · HTML: https://mlsanity.com/paper/1906.06127/

## Key facts

- Cited by 239 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2019.

## Built on by (most cited first, 30 of 239)

- [Detecting Language Model Attacks with Perplexity](https://mlsanity.com/paper/2308.14132/) (2023)
- [LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks](https://mlsanity.com/paper/2412.15204/) (2024)
- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [KLUE: Korean Language Understanding Evaluation](https://mlsanity.com/paper/2105.09680/) (2021)
- [Revisiting Relation Extraction in the era of Large Language Models](https://mlsanity.com/paper/2305.05003/) (2023)
- [BioRED: A Rich Biomedical Relation Extraction Dataset](https://mlsanity.com/paper/2204.04263/) (2022)
- [KoLA: Carefully Benchmarking World Knowledge of Large Language Models](https://mlsanity.com/paper/2306.09296/) (2023)
- [Document-level Relation Extraction as Semantic Segmentation](https://mlsanity.com/paper/2106.03618/) (2021)
- [Entity Structure Within and Throughout: Modeling Mention Dependencies for Document-Level Relation Extraction](https://mlsanity.com/paper/2102.10249/) (2021)
- [Revisiting DocRED -- Addressing the False Negative Problem in Relation Extraction](https://mlsanity.com/paper/2205.12696/) (2022)
- [Structured information extraction from complex scientific text with fine-tuned large language models](https://mlsanity.com/paper/2212.05238/) (2022)
- [Text2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Text](https://mlsanity.com/paper/2308.02357/) (2023)
- [Text-to-Table: A New Way of Information Extraction](https://mlsanity.com/paper/2109.02707/) (2021)
- [Analyzing Dataset Annotation Quality Management in the Wild](https://mlsanity.com/paper/2307.08153/) (2023)
- [Document-Level Relation Extraction with Adaptive Focal Loss and Knowledge Distillation](https://mlsanity.com/paper/2203.10900/) (2022)
- [DREEAM: Guiding Attention with Evidence for Improving Document-Level Relation Extraction](https://mlsanity.com/paper/2302.08675/) (2023)
- [When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks](https://mlsanity.com/paper/2311.08993/) (2023)
- [Instruct and Extract: Instruction Tuning for On-Demand Information Extraction](https://mlsanity.com/paper/2310.16040/) (2023)
- [MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory](https://mlsanity.com/paper/2404.11672/) (2024)
- [An End-to-end Model for Entity-level Relation Extraction using Multi-instance Learning](https://mlsanity.com/paper/2102.05980/) (2021)
- [SIRE: Separate Intra- and Inter-sentential Reasoning for Document-level Relation Extraction](https://mlsanity.com/paper/2106.01709/) (2021)
- [Eider: Empowering Document-level Relation Extraction with Efficient Evidence Extraction and Inference-stage Fusion](https://mlsanity.com/paper/2106.08657/) (2021)
- [Performance Optimization for Semantic Communications: An Attention-based Reinforcement Learning Approach](https://mlsanity.com/paper/2208.08239/) (2022)
- [Revisiting Few-shot Relation Classification: Evaluation Data and Classification Schemes](https://mlsanity.com/paper/2104.08481/) (2021)
- [A Survey on Neural Open Information Extraction: Current Status and Future Directions](https://mlsanity.com/paper/2205.11725/) (2022)
- [Graph Based Network with Contextualized Representations of Turns in Dialogue](https://mlsanity.com/paper/2109.04008/) (2021)
- [A Survey on Extraction of Causal Relations from Natural Language Text](https://mlsanity.com/paper/2101.06426/) (2021)
- [Visual Distant Supervision for Scene Graph Generation](https://mlsanity.com/paper/2103.15365/) (2021)
- [Discriminative Reasoning for Document-level Relation Extraction](https://mlsanity.com/paper/2106.01562/) (2021)
- [A sequence-to-sequence approach for document-level relation extraction](https://mlsanity.com/paper/2204.01098/) (2022)

## Abstract (arXiv)

Multiple entities in a document generally exhibit complex inter-sentence relations, and cannot be well handled by existing relation extraction (RE) methods that typically focus on extracting intra-sentence relations for single entity pairs. In order to accelerate the research on document-level RE, we introduce DocRED, a new dataset constructed from Wikipedia and Wikidata with three features: (1) DocRED annotates both named entities and relations, and is the largest human-annotated dataset for document-level RE from plain text; (2) DocRED requires reading multiple sentences in a document to extract entities and infer their relations by synthesizing all information of the document; (3) along with the human-annotated data, we also offer large-scale distantly supervised data, which enables DocRED to be adopted for both supervised and weakly supervised scenarios. In order to verify the challenges of document-level RE, we implement recent state-of-the-art methods for RE and conduct a thorough evaluation of these methods on DocRED. Empirical results show that DocRED is challenging for existing RE methods, which indicates that document-level RE remains an open problem and requires further efforts. Based on the detailed analysis on the experiments, we discuss multiple promising directions for future research.