# Minimize Exposure Bias of Seq2Seq Models in Joint Entity and Relation Extraction

arXiv: https://arxiv.org/abs/2009.07503 · HTML: https://mlsanity.com/paper/2009.07503/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Sep 2020.

## Built on by (most cited first, 27 of 27)

- [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://mlsanity.com/paper/2210.10341/) (2022)
- [Unified Structure Generation for Universal Information Extraction](https://mlsanity.com/paper/2203.12277/) (2022)
- [Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction](https://mlsanity.com/paper/2106.09232/) (2021)
- [Generative Knowledge Graph Construction: A Review](https://mlsanity.com/paper/2210.12714/) (2022)
- [UniRE: A Unified Label Space for Entity Relation Extraction](https://mlsanity.com/paper/2107.04292/) (2021)
- [A sequence-to-sequence approach for document-level relation extraction](https://mlsanity.com/paper/2204.01098/) (2022)
- [InstructABSA: Instruction Learning for Aspect Based Sentiment Analysis](https://mlsanity.com/paper/2302.08624/) (2023)
- [A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers](https://mlsanity.com/paper/2306.02051/) (2023)
- [Understand the Dynamic World: An End-to-End Knowledge Informed Framework for Open Domain Entity State Tracking](https://mlsanity.com/paper/2304.13854/) (2023)
- [FastRE: Towards Fast Relation Extraction with Convolutional Encoder and Improved Cascade Binary Tagging Framework](https://mlsanity.com/paper/2205.02490/) (2022)
- [Rescue Implicit and Long-tail Cases: Nearest Neighbor Relation Extraction](https://mlsanity.com/paper/2210.11800/) (2022)
- [HySPA: Hybrid Span Generation for Scalable Text-to-Graph Extraction](https://mlsanity.com/paper/2106.15838/) (2021)
- [Wikontic: Constructing Wikidata-Aligned, Ontology-Aware Knowledge Graphs with Large Language Models](https://mlsanity.com/paper/2512.00590/) (2025)
- [A Generative Model for Relation Extraction and Classification](https://mlsanity.com/paper/2202.13229/) (2022)
- [JaMIE: A Pipeline Japanese Medical Information Extraction System](https://mlsanity.com/paper/2111.04261/) (2021)
- [Relation Extraction with Weighted Contrastive Pre-training on Distant Supervision](https://mlsanity.com/paper/2205.08770/) (2022)
- [REKnow: Enhanced Knowledge for Joint Entity and Relation Extraction](https://mlsanity.com/paper/2206.05123/) (2022)
- [Adjacency List Oriented Relational Fact Extraction via Adaptive Multi-task Learning](https://mlsanity.com/paper/2106.01559/) (2021)
- [A Benchmark for End-to-End Zero-Shot Biomedical Relation Extraction with LLMs: Experiments with OpenAI Models](https://mlsanity.com/paper/2504.04083/) (2025)
- [Nearest Neighbor Search over Vectorized Lexico-Syntactic Patterns for Relation Extraction from Financial Documents](https://mlsanity.com/paper/2310.17714/) (2023)
- [A Note on Bias to Complete](https://mlsanity.com/paper/2402.11710/) (2024)
- [HIORE: Leveraging High-order Interactions for Unified Entity Relation Extraction](https://mlsanity.com/paper/2305.04297/) (2023)
- [S2F-NER: Exploring Sequence-to-Forest Generation for Complex Entity Recognition](https://mlsanity.com/paper/2310.18944/) (2023)
- [Exploiting Representation Bias for Data Distillation in Abstractive Text Summarization](https://mlsanity.com/paper/2312.06022/) (2023)
- [Causal Tree Extraction from Medical Case Reports: A Novel Task for Experts-like Text Comprehension](https://mlsanity.com/paper/2503.01302/) (2025)
- [DISCIE -- Discriminative Closed Information Extraction](https://mlsanity.com/paper/2506.16348/) (2025)
- [A Joint Neural Baseline for Concept, Assertion, and Relation Extraction from Clinical Text](https://mlsanity.com/paper/2603.07487/) (2026)

## Abstract (arXiv)

Joint entity and relation extraction aims to extract relation triplets from plain text directly. Prior work leverages Sequence-to-Sequence (Seq2Seq) models for triplet sequence generation. However, Seq2Seq enforces an unnecessary order on the unordered triplets and involves a large decoding length associated with error accumulation. These introduce exposure bias, which may cause the models overfit to the frequent label combination, thus deteriorating the generalization. We propose a novel Sequence-to-Unordered-Multi-Tree (Seq2UMTree) model to minimize the effects of exposure bias by limiting the decoding length to three within a triplet and removing the order among triplets. We evaluate our model on two datasets, DuIE and NYT, and systematically study how exposure bias alters the performance of Seq2Seq models. Experiments show that the state-of-the-art Seq2Seq model overfits to both datasets while Seq2UMTree shows significantly better generalization. Our code is available at https://github.com/WindChimeRan/OpenJERE .