# Enriching Pre-trained Language Model with Entity Information for Relation Classification

arXiv: https://arxiv.org/abs/1905.08284 · HTML: https://mlsanity.com/paper/1905.08284/

## Key facts

- Cited by 82 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 May 2019.

## Built on by (most cited first, 30 of 82)

- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [RadGraph: Extracting Clinical Entities and Relations from Radiology Reports](https://mlsanity.com/paper/2106.14463/) (2021)
- [KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction](https://mlsanity.com/paper/2104.07650/) (2021)
- [WRENCH: A Comprehensive Benchmark for Weak Supervision](https://mlsanity.com/paper/2109.11377/) (2021)
- [A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4](https://mlsanity.com/paper/2310.12321/) (2023)
- [RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction](https://mlsanity.com/paper/2203.09101/) (2022)
- [ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning](https://mlsanity.com/paper/2104.04697/) (2021)
- [Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Review](https://mlsanity.com/paper/2107.02975/) (2021)
- [MatSci-NLP: Evaluating Scientific Language Models on Materials Science Language Tasks Using Text-to-Schema Modeling](https://mlsanity.com/paper/2305.08264/) (2023)
- [An End-to-end Model for Entity-level Relation Extraction using Multi-instance Learning](https://mlsanity.com/paper/2102.05980/) (2021)
- [Summarization as Indirect Supervision for Relation Extraction](https://mlsanity.com/paper/2205.09837/) (2022)
- [Revisiting Large Language Models as Zero-shot Relation Extractors](https://mlsanity.com/paper/2310.05028/) (2023)
- [Looking Beyond IoCs: Automatically Extracting Attack Patterns from External CTI](https://mlsanity.com/paper/2211.01753/) (2022)
- [NSP-BERT: A Prompt-based Few-Shot Learner Through an Original Pre-training Task--Next Sentence Prediction](https://mlsanity.com/paper/2109.03564/) (2021)
- [A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers](https://mlsanity.com/paper/2306.02051/) (2023)
- [BERT Based Clinical Knowledge Extraction for Biomedical Knowledge Graph Construction and Analysis](https://mlsanity.com/paper/2304.10996/) (2023)
- [TAGPRIME: A Unified Framework for Relational Structure Extraction](https://mlsanity.com/paper/2205.12585/) (2022)
- [RE-Matching: A Fine-Grained Semantic Matching Method for Zero-Shot Relation Extraction](https://mlsanity.com/paper/2306.04954/) (2023)
- [Construction of Knowledge Graphs: State and Challenges](https://mlsanity.com/paper/2302.11509/) (2023)
- [ACCoRD: A Multi-Document Approach to Generating Diverse Descriptions of Scientific Concepts](https://mlsanity.com/paper/2205.06982/) (2022)
- [The Leaf Clinical Trials Corpus: a new resource for query generation from clinical trial eligibility criteria](https://mlsanity.com/paper/2207.13757/) (2022)
- [On the Evolution of Knowledge Graphs: A Survey and Perspective](https://mlsanity.com/paper/2310.04835/) (2023)
- [Unleashing the Power of Large Language Models in Zero-shot Relation Extraction via Self-Prompting](https://mlsanity.com/paper/2410.01154/) (2024)
- [Multi-view Inference for Relation Extraction with Uncertain Knowledge](https://mlsanity.com/paper/2104.13579/) (2021)
- [Prompt-based Zero-shot Relation Extraction with Semantic Knowledge Augmentation](https://mlsanity.com/paper/2112.04539/) (2021)
- [Learning to Substitute Ingredients in Recipes](https://mlsanity.com/paper/2302.07960/) (2023)
- [PEANuT: Parameter-Efficient Adaptation with Weight-aware Neural Tweakers](https://mlsanity.com/paper/2410.01870/) (2024)
- [Exploring deep learning methods for recognizing rare diseases and their clinical manifestations from texts](https://mlsanity.com/paper/2109.00343/) (2021)
- [Abstractified Multi-instance Learning (AMIL) for Biomedical Relation Extraction](https://mlsanity.com/paper/2110.12501/) (2021)
- [Knowledge-Augmented Language Models for Cause-Effect Relation Classification](https://mlsanity.com/paper/2112.08615/) (2021)

## Abstract (arXiv)

Relation classification is an important NLP task to extract relations between entities. The state-of-the-art methods for relation classification are primarily based on Convolutional or Recurrent Neural Networks. Recently, the pre-trained BERT model achieves very successful results in many NLP classification / sequence labeling tasks. Relation classification differs from those tasks in that it relies on information of both the sentence and the two target entities. In this paper, we propose a model that both leverages the pre-trained BERT language model and incorporates information from the target entities to tackle the relation classification task. We locate the target entities and transfer the information through the pre-trained architecture and incorporate the corresponding encoding of the two entities. We achieve significant improvement over the state-of-the-art method on the SemEval-2010 task 8 relational dataset.