# Multi-Level Structured Self-Attentions for Distantly Supervised Relation Extraction

arXiv: https://arxiv.org/abs/1809.00699 · HTML: https://mlsanity.com/paper/1809.00699/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Sep 2018.

## Built on by (most cited first, 8 of 8)

- [Attention Mechanism in Neural Networks: Where it Comes and Where it Goes](https://mlsanity.com/paper/2204.13154/) (2022)
- [A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers](https://mlsanity.com/paper/2306.02051/) (2023)
- [Deep Neural Network Based Relation Extraction: An Overview](https://mlsanity.com/paper/2101.01907/) (2021)
- [How Knowledge Graph and Attention Help? A Quantitative Analysis into Bag-level Relation Extraction](https://mlsanity.com/paper/2107.12064/) (2021)
- [A Question-answering Based Framework for Relation Extraction Validation](https://mlsanity.com/paper/2104.02934/) (2021)
- [Two Training Strategies for Improving Relation Extraction over Universal Graph](https://mlsanity.com/paper/2102.06540/) (2021)
- [Distantly Supervised Relation Extraction via Recursive Hierarchy-Interactive Attention and Entity-Order Perception](https://mlsanity.com/paper/2105.08213/) (2021)
- [Cross-stitching Text and Knowledge Graph Encoders for Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2211.01432/) (2022)

## Abstract (arXiv)

Attention mechanisms are often used in deep neural networks for distantly supervised relation extraction (DS-RE) to distinguish valid from noisy instances. However, traditional 1-D vector attention models are insufficient for the learning of different contexts in the selection of valid instances to predict the relationship for an entity pair. To alleviate this issue, we propose a novel multi-level structured (2-D matrix) self-attention mechanism for DS-RE in a multi-instance learning (MIL) framework using bidirectional recurrent neural networks. In the proposed method, a structured word-level self-attention mechanism learns a 2-D matrix where each row vector represents a weight distribution for different aspects of an instance regarding two entities. Targeting the MIL issue, the structured sentence-level attention learns a 2-D matrix where each row vector represents a weight distribution on selection of different valid in-stances. Experiments conducted on two publicly available DS-RE datasets show that the proposed framework with a multi-level structured self-attention mechanism significantly outperform state-of-the-art baselines in terms of PR curves, P@N and F1 measures.