# A Unified Generative Framework for Various NER Subtasks

arXiv: https://arxiv.org/abs/2106.01223 · HTML: https://mlsanity.com/paper/2106.01223/

## Key facts

- Cited by 141 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 70 references that are in the map.
- First posted on arXiv 2 Jun 2021.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [Pre-trained Models for Natural Language Processing: A Survey](https://mlsanity.com/paper/2003.08271/) (2020)
- [Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/1711.02281/) (2017)
- [MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://mlsanity.com/paper/1905.02450/) (2019)
- [LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](https://mlsanity.com/paper/2010.01057/) (2020)
- [Named Entity Recognition with Bidirectional LSTM-CNNs](https://mlsanity.com/paper/1511.08308/) (2015)
- [A Unified MRC Framework for Named Entity Recognition](https://mlsanity.com/paper/1910.11476/) (2019)
- [Named Entity Recognition as Dependency Parsing](https://mlsanity.com/paper/2005.07150/) (2020)
- [A General Framework for Information Extraction using Dynamic Span Graphs](https://mlsanity.com/paper/1904.03296/) (2019)
- [Semi-Supervised Sequence Modeling with Cross-View Training](https://mlsanity.com/paper/1809.08370/) (2018)
- [TENER: Adapting Transformer Encoder for Named Entity Recognition](https://mlsanity.com/paper/1911.04474/) (2019)
- [FLAT: Chinese NER Using Flat-Lattice Transformer](https://mlsanity.com/paper/2004.11795/) (2020)
- [Neural Architectures for Nested NER through Linearization](https://mlsanity.com/paper/1908.06926/) (2019)
- [Does syntax matter? A strong baseline for Aspect-based Sentiment Analysis with RoBERTa](https://mlsanity.com/paper/2104.04986/) (2021)
- [Nested Named Entity Recognition via Second-best Sequence Learning and Decoding](https://mlsanity.com/paper/1909.02250/) (2019)
- [Sequence-to-Nuggets: Nested Entity Mention Detection via Anchor-Region Networks](https://mlsanity.com/paper/1906.03783/) (2019)
- [An Effective Transition-based Model for Discontinuous NER](https://mlsanity.com/paper/2004.13454/) (2020)
- [Neural Segmental Hypergraphs for Overlapping Mention Recognition](https://mlsanity.com/paper/1810.01817/) (2018)
- [Multilingual Language Processing From Bytes](https://mlsanity.com/paper/1512.00103/) (2015)
- [Merge and Label: A novel neural network architecture for nested NER](https://mlsanity.com/paper/1907.00464/) (2019)

## Built on by (most cited first, 30 of 141)

- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Unified Structure Generation for Universal Information Extraction](https://mlsanity.com/paper/2203.12277/) (2022)
- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [Unified Named Entity Recognition as Word-Word Relation Classification](https://mlsanity.com/paper/2112.10070/) (2021)
- [When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities](https://mlsanity.com/paper/2307.16376/) (2023)
- [GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer](https://mlsanity.com/paper/2311.08526/) (2023)
- [BioBART: Pretraining and Evaluation of A Biomedical Generative Language Model](https://mlsanity.com/paper/2204.03905/) (2022)
- [CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors](https://mlsanity.com/paper/2305.05711/) (2023)
- [UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition](https://mlsanity.com/paper/2211.11256/) (2022)
- [MAVE: A Product Dataset for Multi-source Attribute Value Extraction](https://mlsanity.com/paper/2112.08663/) (2021)
- [Packed Levitated Marker for Entity and Relation Extraction](https://mlsanity.com/paper/2109.06067/) (2021)
- [Generative Knowledge Graph Construction: A Review](https://mlsanity.com/paper/2210.12714/) (2022)
- [Text-to-Table: A New Way of Information Extraction](https://mlsanity.com/paper/2109.02707/) (2021)
- [DeepStruct: Pretraining of Language Models for Structure Prediction](https://mlsanity.com/paper/2205.10475/) (2022)
- [LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Model](https://mlsanity.com/paper/2304.06248/) (2023)
- [Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction](https://mlsanity.com/paper/2203.08308/) (2022)
- [Paradigm Shift in Natural Language Processing](https://mlsanity.com/paper/2109.12575/) (2021)
- [InstructionNER: A Multi-Task Instruction-Based Generative Framework for Few-shot NER](https://mlsanity.com/paper/2203.03903/) (2022)
- [CodeKGC: Code Language Model for Generative Knowledge Graph Construction](https://mlsanity.com/paper/2304.09048/) (2023)
- [LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting](https://mlsanity.com/paper/2109.00720/) (2021)
- [DiffuSum: Generation Enhanced Extractive Summarization with Diffusion](https://mlsanity.com/paper/2305.01735/) (2023)
- [Optimizing Bi-Encoder for Named Entity Recognition via Contrastive Learning](https://mlsanity.com/paper/2208.14565/) (2022)
- [NBIAS: A Natural Language Processing Framework for Bias Identification in Text](https://mlsanity.com/paper/2308.01681/) (2023)
- [C-ICL: Contrastive In-context Learning for Information Extraction](https://mlsanity.com/paper/2402.11254/) (2024)
- [Decomposed Meta-Learning for Few-Shot Named Entity Recognition](https://mlsanity.com/paper/2204.05751/) (2022)
- [Evaluation of ChatGPT on Biomedical Tasks: A Zero-Shot Comparison with Fine-Tuned Generative Transformers](https://mlsanity.com/paper/2306.04504/) (2023)
- [Bottom-Up Constituency Parsing and Nested Named Entity Recognition with Pointer Networks](https://mlsanity.com/paper/2110.05419/) (2021)
- [DICE: Data-Efficient Clinical Event Extraction with Generative Models](https://mlsanity.com/paper/2208.07989/) (2022)
- [Parallel Instance Query Network for Named Entity Recognition](https://mlsanity.com/paper/2203.10545/) (2022)

## Abstract (arXiv)

Named Entity Recognition (NER) is the task of identifying spans that represent entities in sentences. Whether the entity spans are nested or discontinuous, the NER task can be categorized into the flat NER, nested NER, and discontinuous NER subtasks. These subtasks have been mainly solved by the token-level sequence labelling or span-level classification. However, these solutions can hardly tackle the three kinds of NER subtasks concurrently. To that end, we propose to formulate the NER subtasks as an entity span sequence generation task, which can be solved by a unified sequence-to-sequence (Seq2Seq) framework. Based on our unified framework, we can leverage the pre-trained Seq2Seq model to solve all three kinds of NER subtasks without the special design of the tagging schema or ways to enumerate spans. We exploit three types of entity representations to linearize entities into a sequence. Our proposed framework is easy-to-implement and achieves state-of-the-art (SoTA) or near SoTA performance on eight English NER datasets, including two flat NER datasets, three nested NER datasets, and three discontinuous NER datasets.