# Entity Structure Within and Throughout: Modeling Mention Dependencies for Document-Level Relation Extraction

arXiv: https://arxiv.org/abs/2102.10249 · HTML: https://mlsanity.com/paper/2102.10249/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 35 references that are in the map.
- First posted on arXiv 20 Feb 2021.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [DocRED: A Large-Scale Document-Level Relation Extraction Dataset](https://mlsanity.com/paper/1906.06127/) (2019)
- [Cross-Sentence N-ary Relation Extraction with Graph LSTMs](https://mlsanity.com/paper/1708.03743/) (2017)
- [Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks](https://mlsanity.com/paper/1810.09536/) (2018)
- [Reasoning with Latent Structure Refinement for Document-Level Relation Extraction](https://mlsanity.com/paper/2005.06312/) (2020)
- [Structured Attention Networks](https://mlsanity.com/paper/1702.00887/) (2017)
- [Coreferential Reasoning Learning for Language Representation](https://mlsanity.com/paper/2004.06870/) (2020)
- [Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs](https://mlsanity.com/paper/1909.00228/) (2019)
- [Distant Supervision for Relation Extraction beyond the Sentence Boundary](https://mlsanity.com/paper/1609.04873/) (2016)
- [Inter-sentence Relation Extraction with Document-level Graph Convolutional Neural Network](https://mlsanity.com/paper/1906.04684/) (2019)
- [Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction](https://mlsanity.com/paper/1802.10569/) (2018)
- [Fine-tune Bert for DocRED with Two-step Process](https://mlsanity.com/paper/1909.11898/) (2019)
- [HIN: Hierarchical Inference Network for Document-Level Relation Extraction](https://mlsanity.com/paper/2003.12754/) (2020)
- [Neural Models for Reasoning over Multiple Mentions using Coreference](https://mlsanity.com/paper/1804.05922/) (2018)
- [N-ary Relation Extraction using Graph State LSTM](https://mlsanity.com/paper/1808.09101/) (2018)
- [Dynamic Entity Representations in Neural Language Models](https://mlsanity.com/paper/1708.00781/) (2017)
- [Extracting Multiple-Relations in One-Pass with Pre-Trained Transformers](https://mlsanity.com/paper/1902.01030/) (2019)
- [Neural Relation Extraction Within and Across Sentence Boundaries](https://mlsanity.com/paper/1810.05102/) (2018)
- [Convolutional neural networks for chemical-disease relation extraction are improved with character-based word embeddings](https://mlsanity.com/paper/1805.10586/) (2018)

## Built on by (most cited first, 30 of 68)

- [GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Records](https://mlsanity.com/paper/2203.03540/) (2022)
- [Document-Level Relation Extraction with Adaptive Focal Loss and Knowledge Distillation](https://mlsanity.com/paper/2203.10900/) (2022)
- [DREEAM: Guiding Attention with Evidence for Improving Document-Level Relation Extraction](https://mlsanity.com/paper/2302.08675/) (2023)
- [WebFormer: The Web-page Transformer for Structure Information Extraction](https://mlsanity.com/paper/2202.00217/) (2022)
- [Link Prediction on N-ary Relational Facts: A Graph-based Approach](https://mlsanity.com/paper/2105.08476/) (2021)
- [A sequence-to-sequence approach for document-level relation extraction](https://mlsanity.com/paper/2204.01098/) (2022)
- [SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction](https://mlsanity.com/paper/2109.12093/) (2021)
- [A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers](https://mlsanity.com/paper/2306.02051/) (2023)
- [Does Recommend-Revise Produce Reliable Annotations? An Analysis on Missing Instances in DocRED](https://mlsanity.com/paper/2204.07980/) (2022)
- [KPI-BERT: A Joint Named Entity Recognition and Relation Extraction Model for Financial Reports](https://mlsanity.com/paper/2208.02140/) (2022)
- [RealCustom++: Representing Images as Real Textual Word for Real-Time Customization](https://mlsanity.com/paper/2408.09744/) (2024)
- [Learning Logic Rules for Document-level Relation Extraction](https://mlsanity.com/paper/2111.05407/) (2021)
- [UniRel: Unified Representation and Interaction for Joint Relational Triple Extraction](https://mlsanity.com/paper/2211.09039/) (2022)
- [Few-Shot Document-Level Relation Extraction](https://mlsanity.com/paper/2205.02048/) (2022)
- [A Unified Positive-Unlabeled Learning Framework for Document-Level Relation Extraction with Different Levels of Labeling](https://mlsanity.com/paper/2210.08709/) (2022)
- [KPI-EDGAR: A Novel Dataset and Accompanying Metric for Relation Extraction from Financial Documents](https://mlsanity.com/paper/2210.09163/) (2022)
- [Document-level Relation Extraction with Cross-sentence Reasoning Graph](https://mlsanity.com/paper/2303.03912/) (2023)
- [RAPL: A Relation-Aware Prototype Learning Approach for Few-Shot Document-Level Relation Extraction](https://mlsanity.com/paper/2310.15743/) (2023)
- [None Class Ranking Loss for Document-Level Relation Extraction](https://mlsanity.com/paper/2205.00476/) (2022)
- [RAAT: Relation-Augmented Attention Transformer for Relation Modeling in Document-Level Event Extraction](https://mlsanity.com/paper/2206.03377/) (2022)
- [LLM with Relation Classifier for Document-Level Relation Extraction](https://mlsanity.com/paper/2408.13889/) (2024)
- [Fine-grained Contrastive Learning for Relation Extraction](https://mlsanity.com/paper/2205.12491/) (2022)
- [Improving Entity Disambiguation by Reasoning over a Knowledge Base](https://mlsanity.com/paper/2207.04106/) (2022)
- [Uncertainty Guided Label Denoising for Document-level Distant Relation Extraction](https://mlsanity.com/paper/2305.11029/) (2023)
- [HistRED: A Historical Document-Level Relation Extraction Dataset](https://mlsanity.com/paper/2307.04285/) (2023)
- [Relation-Specific Attentions over Entity Mentions for Enhanced Document-Level Relation Extraction](https://mlsanity.com/paper/2205.14393/) (2022)
- [CorefDRE: Document-level Relation Extraction with coreference resolution](https://mlsanity.com/paper/2202.10744/) (2022)
- [Transformers as Graph-to-Graph Models](https://mlsanity.com/paper/2310.17936/) (2023)
- [GAMedX: Generative AI-based Medical Entity Data Extractor Using Large Language Models](https://mlsanity.com/paper/2405.20585/) (2024)
- [Document-level Relation Extraction with Context Guided Mention Integration and Inter-pair Reasoning](https://mlsanity.com/paper/2201.04826/) (2022)

## Abstract (arXiv)

Entities, as the essential elements in relation extraction tasks, exhibit certain structure. In this work, we formulate such structure as distinctive dependencies between mention pairs. We then propose SSAN, which incorporates these structural dependencies within the standard self-attention mechanism and throughout the overall encoding stage. Specifically, we design two alternative transformation modules inside each self-attention building block to produce attentive biases so as to adaptively regularize its attention flow. Our experiments demonstrate the usefulness of the proposed entity structure and the effectiveness of SSAN. It significantly outperforms competitive baselines, achieving new state-of-the-art results on three popular document-level relation extraction datasets. We further provide ablation and visualization to show how the entity structure guides the model for better relation extraction. Our code is publicly available.