mlsanityData to 7 Oct 2026

DocRED: A Large-Scale Document-Level Relation Extraction Dataset

arXiv 1906.06127 ↗First posted 14 Jun 2019cs.CLcited by 239

Not on the map. Why?

Abstract (arXiv)

Multiple entities in a document generally exhibit complex inter-sentence relations, and cannot be well handled by existing relation extraction (RE) methods that typically focus on extracting intra-sentence relations for single entity pairs. In order to accelerate the research on document-level RE, we introduce DocRED, a new dataset constructed from Wikipedia and Wikidata with three features: (1) DocRED annotates both named entities and relations, and is the largest human-annotated dataset for document-level RE from plain text; (2) DocRED requires reading multiple sentences in a document to extract entities and infer their relations by synthesizing all information of the document; (3) along with the human-annotated data, we also offer large-scale distantly supervised data, which enables DocRED to be adopted for both supervised and weakly supervised scenarios. In order to verify the challenges of document-level RE, we implement recent state-of-the-art methods for RE and conduct a thorough evaluation of these methods on DocRED. Empirical results show that DocRED is challenging for existing RE methods, which indicates that document-level RE remains an open problem and requires further efforts. Based on the detailed analysis on the experiments, we discuss multiple promising directions for future research.

Key facts. Cited by 239 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 39; 2022: 53; 2023: 56; 2024: 44; 2025: 24; 2026: 23.

239 papers.

Sort:

By area:

  1. 2023Detecting Language Model Attacks with Perplexitycited by 411
  2. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 367
  3. 2023Large Language Models for Generative Information Extraction: A Surveycited by 183
  4. 2021KLUE: Korean Language Understanding Evaluationcited by 113
  5. 2023Revisiting Relation Extraction in the era of Large Language Modelscited by 100
  6. 2023KoLA: Carefully Benchmarking World Knowledge of Large Language Modelscited by 85
  7. 2022BioRED: A Rich Biomedical Relation Extraction Datasetcited by 84
  8. 2021Document-level Relation Extraction as Semantic Segmentationcited by 80
  9. 2021Entity Structure Within and Throughout: Modeling Mention Dependencies for Document-Level Relation Extractioncited by 68
  10. 2022Revisiting DocRED -- Addressing the False Negative Problem in Relation Extractioncited by 57
  11. 2022Structured information extraction from complex scientific text with fine-tuned large language modelscited by 56
  12. 2023Text2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Textcited by 48
38 more
  1. 2021Text-to-Table: A New Way of Information Extractioncited by 47
  2. 2022Document-Level Relation Extraction with Adaptive Focal Loss and Knowledge Distillationcited by 39
  3. 2023Analyzing Dataset Annotation Quality Management in the Wildcited by 39
  4. 2023DREEAM: Guiding Attention with Evidence for Improving Document-Level Relation Extractioncited by 37
  5. 2023When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Taskscited by 37
  6. 2023Instruct and Extract: Instruction Tuning for On-Demand Information Extractioncited by 35
  7. 2024MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memorycited by 35
  8. 2021An End-to-end Model for Entity-level Relation Extraction using Multi-instance Learningcited by 31
  9. 2021SIRE: Separate Intra- and Inter-sentential Reasoning for Document-level Relation Extractioncited by 30
  10. 2021Eider: Empowering Document-level Relation Extraction with Efficient Evidence Extraction and Inference-stage Fusioncited by 28
  11. 2022Performance Optimization for Semantic Communications: An Attention-based Reinforcement Learning Approachcited by 28
  12. 2021Revisiting Few-shot Relation Classification: Evaluation Data and Classification Schemescited by 27
  13. 2022A Survey on Neural Open Information Extraction: Current Status and Future Directionscited by 27
  14. 2021Graph Based Network with Contextualized Representations of Turns in Dialoguecited by 26
  15. 2021A Survey on Extraction of Causal Relations from Natural Language Textcited by 25
  16. 2021Visual Distant Supervision for Scene Graph Generationcited by 25
  17. 2021Discriminative Reasoning for Document-level Relation Extractioncited by 25
  18. 2022A sequence-to-sequence approach for document-level relation extractioncited by 25
  19. 2021SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extractioncited by 23
  20. 2022KALA: Knowledge-Augmented Language Model Adaptationcited by 21
  21. 2023A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontierscited by 21
  22. 2022ERGO: Event Relational Graph Transformer for Document-level Event Causality Identificationcited by 19
  23. 2022Does Recommend-Revise Produce Reliable Annotations? An Analysis on Missing Instances in DocREDcited by 19
  24. 2023Tell2Design: A Dataset for Language-Guided Floor Plan Generationcited by 19
  25. 2022What do You Mean by Relation Extraction? A Survey on Datasets and Study on Scientific Relation Classificationcited by 17
  26. 2021Three Sentences Are All You Need: Local Path Enhanced Document Relation Extractioncited by 16
  27. 2022Document-Level Relation Extraction with Sentences Importance Estimation and Focusingcited by 16
  28. 2024Multi-News+: Cost-efficient Dataset Cleansing via LLM-based Data Annotationcited by 16
  29. 2025Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidencecited by 16
  30. 2021MLBiNet: A Cross-Sentence Collective Event Detection Networkcited by 15
  31. 2021Learning Logic Rules for Document-level Relation Extractioncited by 15
  32. 2021Discovering Drug-Target Interaction Knowledge from Biomedical Literaturecited by 14
  33. 2023TempEL: Linking Dynamically Evolving and Newly Emerging Entitiescited by 14
  34. 2024Consistency Guided Knowledge Retrieval and Denoising in LLMs for Zero-shot Document-level Relation Triplet Extractioncited by 14
  35. 2021NEREL: A Russian Dataset with Nested Named Entities, Relations and Eventscited by 13
  36. 2022A Dataset for Hyper-Relational Extraction and a Cube-Filling Approachcited by 13
  37. 2022Multi-hop Evidence Retrieval for Cross-document Relation Extractioncited by 13
  38. 2023PolyIE: A Dataset of Information Extraction from Polymer Material Scientific Literaturecited by 13

50 most cited of 239 All 239 (JSON)