# Multi-task Transformer with Relation-attention and Type-attention for Named Entity Recognition

arXiv: https://arxiv.org/abs/2303.10870 · HTML: https://mlsanity.com/paper/2303.10870/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 26 references that are in the map.
- First posted on arXiv 20 Mar 2023.

## Builds on (most cited first, 13 of 13)

- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [Named Entity Recognition with Bidirectional LSTM-CNNs](https://mlsanity.com/paper/1511.08308/) (2015)
- [A Unified MRC Framework for Named Entity Recognition](https://mlsanity.com/paper/1910.11476/) (2019)
- [Named Entity Recognition as Dependency Parsing](https://mlsanity.com/paper/2005.07150/) (2020)
- [A Unified Generative Framework for Various NER Subtasks](https://mlsanity.com/paper/2106.01223/) (2021)
- [Are we ready for a new paradigm shift? A Survey on Visual Deep MLP](https://mlsanity.com/paper/2111.04060/) (2021)
- [Neural Architectures for Nested NER through Linearization](https://mlsanity.com/paper/1908.06926/) (2019)
- [Fast and Accurate Entity Recognition with Iterated Dilated Convolutions](https://mlsanity.com/paper/1702.02098/) (2017)
- [An Effective Transition-based Model for Discontinuous NER](https://mlsanity.com/paper/2004.13454/) (2020)
- [Nested Named Entity Recognition with Partially-Observed TreeCRFs](https://mlsanity.com/paper/2012.08478/) (2020)
- [Discontinuous Named Entity Recognition as Maximal Clique Discovery](https://mlsanity.com/paper/2106.00218/) (2021)
- [Combining Spans into Entities: A Neural Two-Stage Approach for Recognizing Discontiguous Entities](https://mlsanity.com/paper/1909.00930/) (2019)

## Built on by (most cited first, 4 of 4)

- [C-ICL: Contrastive In-context Learning for Information Extraction](https://mlsanity.com/paper/2402.11254/) (2024)
- [mCL-NER: Cross-Lingual Named Entity Recognition via Multi-view Contrastive Learning](https://mlsanity.com/paper/2308.09073/) (2023)
- [Contextual Biasing of Named-Entities with Large Language Models](https://mlsanity.com/paper/2309.00723/) (2023)
- [LC-ICL: Label-Guided Contrastive In-Context Learning for Robust Information Extraction](https://mlsanity.com/paper/2606.29407/) (2026)

## Abstract (arXiv)

Named entity recognition (NER) is an important research problem in natural language processing. There are three types of NER tasks, including flat, nested and discontinuous entity recognition. Most previous sequential labeling models are task-specific, while recent years have witnessed the rising of generative models due to the advantage of unifying all NER tasks into the seq2seq model framework. Although achieving promising performance, our pilot studies demonstrate that existing generative models are ineffective at detecting entity boundaries and estimating entity types. This paper proposes a multi-task Transformer, which incorporates an entity boundary detection task into the named entity recognition task. More concretely, we achieve entity boundary detection by classifying the relations between tokens within the sentence. To improve the accuracy of entity-type mapping during decoding, we adopt an external knowledge base to calculate the prior entity-type distributions and then incorporate the information into the model via the self and cross-attention mechanisms. We perform experiments on an extensive set of NER benchmarks, including two flat, three nested, and three discontinuous NER datasets. Experimental results show that our approach considerably improves the generative NER model's performance.