# Span-based Joint Entity and Relation Extraction with Transformer Pre-training

arXiv: https://arxiv.org/abs/1909.07755 · HTML: https://mlsanity.com/paper/1909.07755/

## Key facts

- Cited by 133 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2019.

## Built on by (most cited first, 30 of 133)

- [Structured Prediction as Translation between Augmented Natural Languages](https://mlsanity.com/paper/2101.05779/) (2021)
- [Revisiting Relation Extraction in the era of Large Language Models](https://mlsanity.com/paper/2305.05003/) (2023)
- [DeepStruct: Pretraining of Language Models for Structure Prediction](https://mlsanity.com/paper/2205.10475/) (2022)
- [Autoregressive Structured Prediction with Language Models](https://mlsanity.com/paper/2210.14698/) (2022)
- [A Partition Filter Network for Joint Entity and Relation Extraction](https://mlsanity.com/paper/2108.12202/) (2021)
- [An End-to-end Model for Entity-level Relation Extraction using Multi-instance Learning](https://mlsanity.com/paper/2102.05980/) (2021)
- [Learn from Syntax: Improving Pair-wise Aspect and Opinion Terms Extractionwith Rich Syntactic Knowledge](https://mlsanity.com/paper/2105.02520/) (2021)
- [DiaASQ : A Benchmark of Conversational Aspect-based Sentiment Quadruple Analysis](https://mlsanity.com/paper/2211.05705/) (2022)
- [A Trigger-Sense Memory Flow Framework for Joint Entity and Relation Extraction](https://mlsanity.com/paper/2101.10213/) (2021)
- [Boundary Smoothing for Named Entity Recognition](https://mlsanity.com/paper/2204.12031/) (2022)
- [A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers](https://mlsanity.com/paper/2306.02051/) (2023)
- [Joint Biomedical Entity and Relation Extraction with Knowledge-Enhanced Collective Inference](https://mlsanity.com/paper/2105.13456/) (2021)
- [KPI-BERT: A Joint Named Entity Recognition and Relation Extraction Model for Financial Reports](https://mlsanity.com/paper/2208.02140/) (2022)
- [A Novel Global Feature-Oriented Relational Triple Extraction Model based on Table Filling](https://mlsanity.com/paper/2109.06705/) (2021)
- [Global Pointer: Novel Efficient Span-based Approach for Named Entity Recognition](https://mlsanity.com/paper/2208.03054/) (2022)
- [A Simple but Effective Bidirectional Framework for Relational Triple Extraction](https://mlsanity.com/paper/2112.04940/) (2021)
- [Efficient Knowledge Infusion via KG-LLM Alignment](https://mlsanity.com/paper/2406.03746/) (2024)
- [GLiREL -- Generalist Model for Zero-Shot Relation Extraction](https://mlsanity.com/paper/2501.03172/) (2025)
- [Distilling Large Language Models for Biomedical Knowledge Extraction: A Case Study on Adverse Drug Events](https://mlsanity.com/paper/2307.06439/) (2023)
- [Joint Entity and Relation Extraction with Span Pruning and Hypergraph Neural Networks](https://mlsanity.com/paper/2310.17238/) (2023)
- [R1-RE: Cross-Domain Relation Extraction with RLVR](https://mlsanity.com/paper/2507.04642/) (2025)
- [Extracting Fine-Grained Knowledge Graphs of Scientific Claims: Dataset and Transformer-Based Results](https://mlsanity.com/paper/2109.10453/) (2021)
- [Towards Relation Extraction From Speech](https://mlsanity.com/paper/2210.08759/) (2022)
- [Jointly Extracting Interventions, Outcomes, and Findings from RCT Reports with LLMs](https://mlsanity.com/paper/2305.03642/) (2023)
- [On the Evolution of Knowledge Graphs: A Survey and Perspective](https://mlsanity.com/paper/2310.04835/) (2023)
- [Imposing Relation Structure in Language-Model Embeddings Using Contrastive Learning](https://mlsanity.com/paper/2109.00840/) (2021)
- [KPI-EDGAR: A Novel Dataset and Accompanying Metric for Relation Extraction from Financial Documents](https://mlsanity.com/paper/2210.09163/) (2022)
- [A Data Bootstrapping Recipe for Low Resource Multilingual Relation Classification](https://mlsanity.com/paper/2110.09570/) (2021)
- [WebIE: Faithful and Robust Information Extraction on the Web](https://mlsanity.com/paper/2305.14293/) (2023)
- [Prompt-based Extraction of Social Determinants of Health Using Few-shot Learning](https://mlsanity.com/paper/2306.07170/) (2023)

## Abstract (arXiv)

We introduce SpERT, an attention model for span-based joint entity and relation extraction. Our key contribution is a light-weight reasoning on BERT embeddings, which features entity recognition and filtering, as well as relation classification with a localized, marker-free context representation. The model is trained using strong within-sentence negative samples, which are efficiently extracted in a single BERT pass. These aspects facilitate a search over all spans in the sentence. In ablation studies, we demonstrate the benefits of pre-training, strong negative sampling and localized context. Our model outperforms prior work by up to 2.6% F1 score on several datasets for joint entity and relation extraction.