# Self-Attention Enhanced Selective Gate with Entity-Aware Embedding for Distantly Supervised Relation Extraction

arXiv: https://arxiv.org/abs/1911.11899 · HTML: https://mlsanity.com/paper/1911.11899/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Nov 2019.

## Built on by (most cited first, 18 of 18)

- [WebFormer: The Web-page Transformer for Structure Information Extraction](https://mlsanity.com/paper/2202.00217/) (2022)
- [Localized Concept Erasure for Text-to-Image Diffusion Models Using Training-Free Gated Low-Rank Adaptation](https://mlsanity.com/paper/2503.12356/) (2025)
- [A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers](https://mlsanity.com/paper/2306.02051/) (2023)
- [Concept Pinpoint Eraser for Text-to-image Diffusion Models via Residual Attention Gate](https://mlsanity.com/paper/2506.22806/) (2025)
- [Deep Neural Approaches to Relation Triplets Extraction: A Comprehensive Survey](https://mlsanity.com/paper/2103.16929/) (2021)
- [SENT: Sentence-level Distant Relation Extraction via Negative Training](https://mlsanity.com/paper/2106.11566/) (2021)
- [Distantly-Supervised Long-Tailed Relation Extraction Using Constraint Graphs](https://mlsanity.com/paper/2105.11225/) (2021)
- [Distantly Supervised Relation Extraction with Sentence Reconstruction and Knowledge Base Priors](https://mlsanity.com/paper/2104.08225/) (2021)
- [How Knowledge Graph and Attention Help? A Quantitative Analysis into Bag-level Relation Extraction](https://mlsanity.com/paper/2107.12064/) (2021)
- [Uncertainty Guided Label Denoising for Document-level Distant Relation Extraction](https://mlsanity.com/paper/2305.11029/) (2023)
- [An Overview of Distant Supervision for Relation Extraction with a Focus on Denoising and Pre-training Methods](https://mlsanity.com/paper/2207.08286/) (2022)
- [Enhancing Contrastive Learning with Noise-Guided Attack: Towards Continual Relation Extraction in the Wild](https://mlsanity.com/paper/2305.07085/) (2023)
- [Mathematical Formalism for Memory Compression in Selective State Space Models](https://mlsanity.com/paper/2410.03158/) (2024)
- [Distantly Supervised Relation Extraction via Recursive Hierarchy-Interactive Attention and Entity-Order Perception](https://mlsanity.com/paper/2105.08213/) (2021)
- [Cross-stitching Text and Knowledge Graph Encoders for Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2211.01432/) (2022)
- [A Sample-Based Training Method for Distantly Supervised Relation Extraction with Pre-Trained Transformers](https://mlsanity.com/paper/2104.07512/) (2021)
- [Hierarchical Relation-Guided Type-Sentence Alignment for Long-Tail Relation Extraction with Distant Supervision](https://mlsanity.com/paper/2109.09036/) (2021)
- [OASYS: Domain-Agnostic Automated System for Constructing Knowledge Base from Unstructured Text](https://mlsanity.com/paper/2207.07597/) (2022)

## Abstract (arXiv)

Distantly supervised relation extraction intrinsically suffers from noisy labels due to the strong assumption of distant supervision. Most prior works adopt a selective attention mechanism over sentences in a bag to denoise from wrongly labeled data, which however could be incompetent when there is only one sentence in a bag. In this paper, we propose a brand-new light-weight neural framework to address the distantly supervised relation extraction problem and alleviate the defects in previous selective attention framework. Specifically, in the proposed framework, 1) we use an entity-aware word embedding method to integrate both relative position information and head/tail entity embeddings, aiming to highlight the essence of entities for this task; 2) we develop a self-attention mechanism to capture the rich contextual dependencies as a complement for local dependencies captured by piecewise CNN; and 3) instead of using selective attention, we design a pooling-equipped gate, which is based on rich contextual representations, as an aggregator to generate bag-level representation for final relation classification. Compared to selective attention, one major advantage of the proposed gating mechanism is that, it performs stably and promisingly even if only one sentence appears in a bag and thus keeps the consistency across all training examples. The experiments on NYT dataset demonstrate that our approach achieves a new state-of-the-art performance in terms of both AUC and top-n precision metrics.