# End-to-end Named Entity Recognition and Relation Extraction using Pre-trained Language Models

arXiv: https://arxiv.org/abs/1912.13415 · HTML: https://mlsanity.com/paper/1912.13415/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Dec 2019.

## Built on by (most cited first, 11 of 11)

- [Pre-trained Language Models in Biomedical Domain: A Systematic Survey](https://mlsanity.com/paper/2110.05006/) (2021)
- [Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling](https://mlsanity.com/paper/2306.11489/) (2023)
- [A sequence-to-sequence approach for document-level relation extraction](https://mlsanity.com/paper/2204.01098/) (2022)
- [Distantly Supervised Named Entity Recognition via Confidence-Based Multi-Class Positive and Unlabeled Learning](https://mlsanity.com/paper/2204.09589/) (2022)
- [KPI-BERT: A Joint Named Entity Recognition and Relation Extraction Model for Financial Reports](https://mlsanity.com/paper/2208.02140/) (2022)
- [Mining for Species, Locations, Habitats, and Ecosystems from Scientific Papers in Invasion Biology: A Large-Scale Exploratory Study with Large Language Models](https://mlsanity.com/paper/2501.18287/) (2025)
- [Few Dimensions are Enough: Fine-tuning BERT with Selected Dimensions Revealed Its Redundant Nature](https://mlsanity.com/paper/2504.04966/) (2025)
- [Providing Insights for Open-Response Surveys via End-to-End Context-Aware Clustering](https://mlsanity.com/paper/2203.01294/) (2022)
- [WCL-BBCD: A Contrastive Learning and Knowledge Graph Approach to Named Entity Recognition](https://mlsanity.com/paper/2203.06925/) (2022)
- [Controlled Randomness Improves the Performance of Transformer Models](https://mlsanity.com/paper/2310.13526/) (2023)
- [Clinical Trials Ontology Engineering with Large Language Models](https://mlsanity.com/paper/2412.14387/) (2024)

## Abstract (arXiv)

Named entity recognition (NER) and relation extraction (RE) are two important tasks in information extraction and retrieval (IE & IR). Recent work has demonstrated that it is beneficial to learn these tasks jointly, which avoids the propagation of error inherent in pipeline-based systems and improves performance. However, state-of-the-art joint models typically rely on external natural language processing (NLP) tools, such as dependency parsers, limiting their usefulness to domains (e.g. news) where those tools perform well. The few neural, end-to-end models that have been proposed are trained almost completely from scratch. In this paper, we propose a neural, end-to-end model for jointly extracting entities and their relations which does not rely on external NLP tools and which integrates a large, pre-trained language model. Because the bulk of our model's parameters are pre-trained and we eschew recurrence for self-attention, our model is fast to train. On 5 datasets across 3 domains, our model matches or exceeds state-of-the-art performance, sometimes by a large margin.