# SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction

arXiv: https://arxiv.org/abs/2109.12093 · HTML: https://mlsanity.com/paper/2109.12093/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 54 references that are in the map.
- First posted on arXiv 24 Sep 2021.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [Pre-Trained Models: Past, Present and Future](https://mlsanity.com/paper/2106.07139/) (2021)
- [DocRED: A Large-Scale Document-Level Relation Extraction Dataset](https://mlsanity.com/paper/1906.06127/) (2019)
- [Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling](https://mlsanity.com/paper/2010.11304/) (2020)
- [Reasoning with Latent Structure Refinement for Document-Level Relation Extraction](https://mlsanity.com/paper/2005.06312/) (2020)
- [Double Graph Based Reasoning for Document-level Relation Extraction](https://mlsanity.com/paper/2009.13752/) (2020)
- [Attention Guided Graph Convolutional Networks for Relation Extraction](https://mlsanity.com/paper/1906.07510/) (2019)
- [Coreferential Reasoning Learning for Language Representation](https://mlsanity.com/paper/2004.06870/) (2020)
- [Document-level Relation Extraction as Semantic Segmentation](https://mlsanity.com/paper/2106.03618/) (2021)
- [Learning from Context or Names? An Empirical Study on Neural Relation Extraction](https://mlsanity.com/paper/2010.01923/) (2020)
- [Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs](https://mlsanity.com/paper/1909.00228/) (2019)
- [Entity Structure Within and Throughout: Modeling Mention Dependencies for Document-Level Relation Extraction](https://mlsanity.com/paper/2102.10249/) (2021)
- [SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals](https://mlsanity.com/paper/1911.10422/) (2019)
- [OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction](https://mlsanity.com/paper/1909.13078/) (2019)
- [Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks](https://mlsanity.com/paper/1903.01306/) (2019)
- [Learning Deep Bilinear Transformation for Fine-grained Image Representation](https://mlsanity.com/paper/1911.03621/) (2019)
- [Document-Level N-ary Relation Extraction with Multiscale Representation Learning](https://mlsanity.com/paper/1904.02347/) (2019)
- [Inter-sentence Relation Extraction with Document-level Graph Convolutional Neural Network](https://mlsanity.com/paper/1906.04684/) (2019)
- [Reinforcement Learning for Relation Classification from Noisy Data](https://mlsanity.com/paper/1808.08013/) (2018)
- [Relation Extraction : A Survey](https://mlsanity.com/paper/1712.05191/) (2017)
- [Global-to-Local Neural Networks for Document-Level Relation Extraction](https://mlsanity.com/paper/2009.10359/) (2020)
- [Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction](https://mlsanity.com/paper/1802.10569/) (2018)
- [Fine-tune Bert for DocRED with Two-step Process](https://mlsanity.com/paper/1909.11898/) (2019)

## Built on by (most cited first, 23 of 23)

- [Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis](https://mlsanity.com/paper/2210.04714/) (2022)
- [DREEAM: Guiding Attention with Evidence for Improving Document-Level Relation Extraction](https://mlsanity.com/paper/2302.08675/) (2023)
- [Few-Shot Document-Level Relation Extraction](https://mlsanity.com/paper/2205.02048/) (2022)
- [RAPL: A Relation-Aware Prototype Learning Approach for Few-Shot Document-Level Relation Extraction](https://mlsanity.com/paper/2310.15743/) (2023)
- [Fine-grained Contrastive Learning for Relation Extraction](https://mlsanity.com/paper/2205.12491/) (2022)
- [HistRED: A Historical Document-Level Relation Extraction Dataset](https://mlsanity.com/paper/2307.04285/) (2023)
- [Pareto Optimal Learning for Estimating Large Language Model Errors](https://mlsanity.com/paper/2306.16564/) (2023)
- [Towards Integration of Discriminability and Robustness for Document-Level Relation Extraction](https://mlsanity.com/paper/2304.00824/) (2023)
- [Rethinking Document-Level Relation Extraction: A Reality Check](https://mlsanity.com/paper/2306.08953/) (2023)
- [A Comprehensive Survey of Document-level Relation Extraction (2016-2023)](https://mlsanity.com/paper/2309.16396/) (2023)
- [DORE: Document Ordered Relation Extraction based on Generative Framework](https://mlsanity.com/paper/2210.16064/) (2022)
- [Did the Models Understand Documents? Benchmarking Models for Language Understanding in Document-Level Relation Extraction](https://mlsanity.com/paper/2306.11386/) (2023)
- [A Survey of Document-Level Information Extraction](https://mlsanity.com/paper/2309.13249/) (2023)
- [Anaphor Assisted Document-Level Relation Extraction](https://mlsanity.com/paper/2310.18604/) (2023)
- [Document-Level In-Context Few-Shot Relation Extraction via Pre-Trained Language Models](https://mlsanity.com/paper/2310.11085/) (2023)
- [GADePo: Graph-Assisted Declarative Pooling Transformers for Document-Level Relation Extraction](https://mlsanity.com/paper/2308.14423/) (2023)
- [On the Robustness of Document-Level Relation Extraction Models to Entity Name Variations](https://mlsanity.com/paper/2406.07444/) (2024)
- [GEGA: Graph Convolutional Networks and Evidence Retrieval Guided Attention for Enhanced Document-level Relation Extraction](https://mlsanity.com/paper/2407.21384/) (2024)
- [Do the Benefits of Joint Models for Relation Extraction Extend to Document-level Tasks?](https://mlsanity.com/paper/2310.00696/) (2023)
- [Graph-DPEP: Decomposed Plug and Ensemble Play for Few-Shot Document Relation Extraction with Graph-of-Thoughts Reasoning](https://mlsanity.com/paper/2411.02864/) (2024)
- [CDER: Collaborative Evidence Retrieval for Document-level Relation Extraction](https://mlsanity.com/paper/2504.06529/) (2025)
- [Multi-Relation Extraction in Entity Pairs using Global Context](https://mlsanity.com/paper/2507.22926/) (2025)
- [DOREMI: Optimizing Long Tail Predictions in Document-Level Relation Extraction](https://mlsanity.com/paper/2601.11190/) (2026)

## Abstract (arXiv)

Stepping from sentence-level to document-level, the research on relation extraction (RE) confronts increasing text length and more complicated entity interactions. Consequently, it is more challenging to encode the key information sources--relevant contexts and entity types. However, existing methods only implicitly learn to model these critical information sources while being trained for RE. As a result, they suffer the problems of ineffective supervision and uninterpretable model predictions. In contrast, we propose to explicitly teach the model to capture relevant contexts and entity types by supervising and augmenting intermediate steps (SAIS) for RE. Based on a broad spectrum of carefully designed tasks, our proposed SAIS method not only extracts relations of better quality due to more effective supervision, but also retrieves the corresponding supporting evidence more accurately so as to enhance interpretability. By assessing model uncertainty, SAIS further boosts the performance via evidence-based data augmentation and ensemble inference while reducing the computational cost. Eventually, SAIS delivers state-of-the-art RE results on three benchmarks (DocRED, CDR, and GDA) and outperforms the runner-up by 5.04% relatively in F1 score in evidence retrieval on DocRED.