# A Unified Positive-Unlabeled Learning Framework for Document-Level Relation Extraction with Different Levels of Labeling arXiv: https://arxiv.org/abs/2210.08709 ยท HTML: https://mlsanity.com/paper/2210.08709/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Builds on 31 of its 47 references that are in the map. - First posted on arXiv 17 Oct 2022. ## Builds on (most cited first, 30 of 31) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019) - [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017) - [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020) - [DocRED: A Large-Scale Document-Level Relation Extraction Dataset](https://mlsanity.com/paper/1906.06127/) (2019) - [Positive-Unlabeled Learning with Non-Negative Risk Estimator](https://mlsanity.com/paper/1703.00593/) (2017) - [End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures](https://mlsanity.com/paper/1601.00770/) (2016) - [Graph Convolution over Pruned Dependency Trees Improves Relation Extraction](https://mlsanity.com/paper/1809.10185/) (2018) - [Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Tasks](https://mlsanity.com/paper/2006.07322/) (2020) - [Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling](https://mlsanity.com/paper/2010.11304/) (2020) - [Reasoning with Latent Structure Refinement for Document-Level Relation Extraction](https://mlsanity.com/paper/2005.06312/) (2020) - [Double Graph Based Reasoning for Document-level Relation Extraction](https://mlsanity.com/paper/2009.13752/) (2020) - [Document-level Relation Extraction as Semantic Segmentation](https://mlsanity.com/paper/2106.03618/) (2021) - [Entity Structure Within and Throughout: Modeling Mention Dependencies for Document-Level Relation Extraction](https://mlsanity.com/paper/2102.10249/) (2021) - [Mixture Proportion Estimation and PU Learning: A Modern Approach](https://mlsanity.com/paper/2111.00980/) (2021) - [Distantly Supervised Named Entity Recognition using Positive-Unlabeled Learning](https://mlsanity.com/paper/1906.01378/) (2019) - [Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction](https://mlsanity.com/paper/1802.10569/) (2018) - [Document-Level Relation Extraction with Reconstruction](https://mlsanity.com/paper/2012.11384/) (2020) - [Improving Pairwise Ranking for Multi-label Image Classification](https://mlsanity.com/paper/1704.03135/) (2017) - [Document-Level Relation Extraction with Adaptive Focal Loss and Knowledge Distillation](https://mlsanity.com/paper/2203.10900/) (2022) - [SIRE: Separate Intra- and Inter-sentential Reasoning for Document-level Relation Extraction](https://mlsanity.com/paper/2106.01709/) (2021) - [Distantly Supervised Named Entity Recognition via Confidence-Based Multi-Class Positive and Unlabeled Learning](https://mlsanity.com/paper/2204.09589/) (2022) - [Does Recommend-Revise Produce Reliable Annotations? An Analysis on Missing Instances in DocRED](https://mlsanity.com/paper/2204.07980/) (2022) - [Document-Level Relation Extraction with Sentences Importance Estimation and Focusing](https://mlsanity.com/paper/2204.12679/) (2022) - [Positive-Unlabeled Learning with Adversarial Data Augmentation for Knowledge Graph Completion](https://mlsanity.com/paper/2205.00904/) (2022) - [A Distant Supervision Corpus for Extracting Biomedical Relationships Between Chemicals, Diseases and Genes](https://mlsanity.com/paper/2204.06584/) (2022) - [None Class Ranking Loss for Document-Level Relation Extraction](https://mlsanity.com/paper/2205.00476/) (2022) - [Positive-Unlabeled Classification under Class Prior Shift and Asymmetric Error](https://mlsanity.com/paper/1809.07011/) (2018) - [Improving Neural Relation Extraction with Positive and Unlabeled Learning](https://mlsanity.com/paper/1911.12556/) (2019) - [Positive-Unlabeled Classification under Class-Prior Shift: A Prior-invariant Approach Based on Density Ratio Estimation](https://mlsanity.com/paper/2107.05045/) (2021) ## Built on by (most cited first, 9 of 9) - [A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers](https://mlsanity.com/paper/2306.02051/) (2023) - [Uncertainty Guided Label Denoising for Document-level Distant Relation Extraction](https://mlsanity.com/paper/2305.11029/) (2023) - [A Positive-Unlabeled Metric Learning Framework for Document-Level Relation Extraction with Incomplete Labeling](https://mlsanity.com/paper/2306.14806/) (2023) - [PromptRE: Weakly-Supervised Document-Level Relation Extraction via Prompting-Based Data Programming](https://mlsanity.com/paper/2310.09265/) (2023) - [Class-Adaptive Self-Training for Relation Extraction with Incompletely Annotated Training Data](https://mlsanity.com/paper/2306.09697/) (2023) - [TTM-RE: Memory-Augmented Document-Level Relation Extraction](https://mlsanity.com/paper/2406.05906/) (2024) - [Combining Supervised Learning and Reinforcement Learning for Multi-Label Classification Tasks with Partial Labels](https://mlsanity.com/paper/2406.16293/) (2024) - [KnowRA: Knowledge Retrieval Augmented Method for Document-level Relation Extraction with Comprehensive Reasoning Abilities](https://mlsanity.com/paper/2501.00571/) (2024) - [Multi-Relation Extraction in Entity Pairs using Global Context](https://mlsanity.com/paper/2507.22926/) (2025) ## Abstract (arXiv) Document-level relation extraction (RE) aims to identify relations between entities across multiple sentences. Most previous methods focused on document-level RE under full supervision. However, in real-world scenario, it is expensive and difficult to completely label all relations in a document because the number of entity pairs in document-level RE grows quadratically with the number of entities. To solve the common incomplete labeling problem, we propose a unified positive-unlabeled learning framework - shift and squared ranking loss positive-unlabeled (SSR-PU) learning. We use positive-unlabeled (PU) learning on document-level RE for the first time. Considering that labeled data of a dataset may lead to prior shift of unlabeled data, we introduce a PU learning under prior shift of training data. Also, using none-class score as an adaptive threshold, we propose squared ranking loss and prove its Bayesian consistency with multi-label ranking metrics. Extensive experiments demonstrate that our method achieves an improvement of about 14 F1 points relative to the previous baseline with incomplete labeling. In addition, it outperforms previous state-of-the-art results under both fully supervised and extremely unlabeled settings as well.