# Distantly Supervised Relation Extraction with Sentence Reconstruction and Knowledge Base Priors

arXiv: https://arxiv.org/abs/2104.08225 · HTML: https://mlsanity.com/paper/2104.08225/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 64 references that are in the map.
- First posted on arXiv 16 Apr 2021.

## Builds on (most cited first, 30 of 35)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation](https://mlsanity.com/paper/1911.06136/) (2019)
- [Knowledge Enhanced Contextual Word Representations](https://mlsanity.com/paper/1909.04164/) (2019)
- [VAE with a VampPrior](https://mlsanity.com/paper/1705.07120/) (2017)
- [Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders](https://mlsanity.com/paper/1611.02648/) (2016)
- [Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space](https://mlsanity.com/paper/2004.04092/) (2020)
- [Effective Modeling of Encoder-Decoder Architecture for Joint Entity and Relation Extraction](https://mlsanity.com/paper/1911.09886/) (2019)
- [Efficient softmax approximation for GPUs](https://mlsanity.com/paper/1609.04309/) (2016)
- [DGL-KE: Training Knowledge Graph Embeddings at Scale](https://mlsanity.com/paper/2004.08532/) (2020)
- [OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction](https://mlsanity.com/paper/1909.13078/) (2019)
- [Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks](https://mlsanity.com/paper/1903.01306/) (2019)
- [Resampled Priors for Variational Autoencoders](https://mlsanity.com/paper/1810.11428/) (2018)
- [More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction](https://mlsanity.com/paper/2004.03186/) (2020)
- [RESIDE: Improving Distantly-Supervised Neural Relation Extraction using Side Information](https://mlsanity.com/paper/1812.04361/) (2018)
- [Reinforcement Learning for Relation Classification from Noisy Data](https://mlsanity.com/paper/1808.08013/) (2018)
- [Topic-Guided Variational Autoencoders for Text Generation](https://mlsanity.com/paper/1903.07137/) (2019)
- [Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction](https://mlsanity.com/paper/1906.08646/) (2019)
- [Robust Distant Supervision Relation Extraction via Deep Reinforcement Learning](https://mlsanity.com/paper/1805.09927/) (2018)
- [Distant Supervision Relation Extraction with Intra-Bag and Inter-Bag Attentions](https://mlsanity.com/paper/1904.00143/) (2019)
- [DSGAN: Generative Adversarial Training for Distant Supervision Relation Extraction](https://mlsanity.com/paper/1805.09929/) (2018)
- [Nonparametric Variational Auto-encoders for Hierarchical Representation Learning](https://mlsanity.com/paper/1703.07027/) (2017)
- [Connecting Language and Knowledge Bases with Embedding Models for Relation Extraction](https://mlsanity.com/paper/1307.7973/) (2013)
- [Self-Attention Enhanced Selective Gate with Entity-Aware Embedding for Distantly Supervised Relation Extraction](https://mlsanity.com/paper/1911.11899/) (2019)
- [Cross-relation Cross-bag Attention for Distantly-supervised Relation Extraction](https://mlsanity.com/paper/1812.10604/) (2018)
- [Connecting Language and Knowledge with Heterogeneous Representations for Neural Relation Extraction](https://mlsanity.com/paper/1903.10126/) (2019)
- [Improving Distantly Supervised Relation Extraction with Neural Noise Converter and Conditional Optimal Selector](https://mlsanity.com/paper/1811.05616/) (2018)
- [Seq2RDF: An end-to-end application for deriving Triples from Natural Language Text](https://mlsanity.com/paper/1807.01763/) (2018)
- [Combining Distant and Direct Supervision for Neural Relation Extraction](https://mlsanity.com/paper/1810.12956/) (2018)

## Built on by (most cited first, 8 of 8)

- [Event Temporal Relation Extraction with Bayesian Translational Model](https://mlsanity.com/paper/2302.04985/) (2023)
- [Distantly-Supervised Long-Tailed Relation Extraction Using Constraint Graphs](https://mlsanity.com/paper/2105.11225/) (2021)
- [AGI for Agriculture](https://mlsanity.com/paper/2304.06136/) (2023)
- [HiCLRE: A Hierarchical Contrastive Learning Framework for Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2202.13352/) (2022)
- [Knowing False Negatives: An Adversarial Training Method for Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2109.02099/) (2021)
- [Span-based Named Entity Recognition by Generating and Compressing Information](https://mlsanity.com/paper/2302.05392/) (2023)
- [Augmenting Task-Oriented Dialogue Systems with Relation Extraction](https://mlsanity.com/paper/2210.13344/) (2022)
- [Knowledge-Enhanced Relation Extraction Dataset](https://mlsanity.com/paper/2210.11231/) (2022)

## Abstract (arXiv)

We propose a multi-task, probabilistic approach to facilitate distantly supervised relation extraction by bringing closer the representations of sentences that contain the same Knowledge Base pairs. To achieve this, we bias the latent space of sentences via a Variational Autoencoder (VAE) that is trained jointly with a relation classifier. The latent code guides the pair representations and influences sentence reconstruction. Experimental results on two datasets created via distant supervision indicate that multi-task learning results in performance benefits. Additional exploration of employing Knowledge Base priors into the VAE reveals that the sentence space can be shifted towards that of the Knowledge Base, offering interpretability and further improving results.