# Modular Self-Supervision for Document-Level Relation Extraction

arXiv: https://arxiv.org/abs/2109.05362 · HTML: https://mlsanity.com/paper/2109.05362/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 41 references that are in the map.
- First posted on arXiv 11 Sep 2021.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [Neural Module Networks](https://mlsanity.com/paper/1511.02799/) (2015)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [DocRED: A Large-Scale Document-Level Relation Extraction Dataset](https://mlsanity.com/paper/1906.06127/) (2019)
- [End-to-End Differentiable Proving](https://mlsanity.com/paper/1705.11040/) (2017)
- [Higher-order Coreference Resolution with Coarse-to-fine Inference](https://mlsanity.com/paper/1804.05392/) (2018)
- [Cross-Sentence N-ary Relation Extraction with Graph LSTMs](https://mlsanity.com/paper/1708.03743/) (2017)
- [Double Graph Based Reasoning for Document-level Relation Extraction](https://mlsanity.com/paper/2009.13752/) (2020)
- [Coreferential Reasoning Learning for Language Representation](https://mlsanity.com/paper/2004.06870/) (2020)
- [Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs](https://mlsanity.com/paper/1909.00228/) (2019)
- [Distant Supervision for Relation Extraction beyond the Sentence Boundary](https://mlsanity.com/paper/1609.04873/) (2016)
- [Document-Level N-ary Relation Extraction with Multiscale Representation Learning](https://mlsanity.com/paper/1904.02347/) (2019)
- [Joint Extraction of Events and Entities within a Document Context](https://mlsanity.com/paper/1609.03632/) (2016)
- [Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction](https://mlsanity.com/paper/1802.10569/) (2018)
- [Three Sentences Are All You Need: Local Path Enhanced Document Relation Extraction](https://mlsanity.com/paper/2106.01793/) (2021)
- [Deep Probabilistic Logic: A Unifying Framework for Indirect Supervision](https://mlsanity.com/paper/1808.08485/) (2018)

## Built on by (most cited first, 4 of 4)

- [A sequence-to-sequence approach for document-level relation extraction](https://mlsanity.com/paper/2204.01098/) (2022)
- [Rethinking Document-Level Relation Extraction: A Reality Check](https://mlsanity.com/paper/2306.08953/) (2023)
- [Towards Structuring Real-World Data at Scale: Deep Learning for Extracting Key Oncology Information from Clinical Text with Patient-Level Supervision](https://mlsanity.com/paper/2203.10442/) (2022)
- [Multi-Relation Extraction in Entity Pairs using Global Context](https://mlsanity.com/paper/2507.22926/) (2025)

## Abstract (arXiv)

Extracting relations across large text spans has been relatively underexplored in NLP, but it is particularly important for high-value domains such as biomedicine, where obtaining high recall of the latest findings is crucial for practical applications. Compared to conventional information extraction confined to short text spans, document-level relation extraction faces additional challenges in both inference and learning. Given longer text spans, state-of-the-art neural architectures are less effective and task-specific self-supervision such as distant supervision becomes very noisy. In this paper, we propose decomposing document-level relation extraction into relation detection and argument resolution, taking inspiration from Davidsonian semantics. This enables us to incorporate explicit discourse modeling and leverage modular self-supervision for each sub-problem, which is less noise-prone and can be further refined end-to-end via variational EM. We conduct a thorough evaluation in biomedical machine reading for precision oncology, where cross-paragraph relation mentions are prevalent. Our method outperforms prior state of the art, such as multi-scale learning and graph neural networks, by over 20 absolute F1 points. The gain is particularly pronounced among the most challenging relation instances whose arguments never co-occur in a paragraph.