# TOE: A Grid-Tagging Discontinuous NER Model Enhanced by Embedding Tag/Word Relations and More Fine-Grained Tags arXiv: https://arxiv.org/abs/2211.00684 ยท HTML: https://mlsanity.com/paper/2211.00684/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Builds on 32 of its 59 references that are in the map. - First posted on arXiv 1 Nov 2022. ## Builds on (most cited first, 30 of 32) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019) - [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016) - [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018) - [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019) - [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019) - [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016) - [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015) - [Circle Loss: A Unified Perspective of Pair Similarity Optimization](https://mlsanity.com/paper/2002.10857/) (2020) - [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016) - [Deep Biaffine Attention for Neural Dependency Parsing](https://mlsanity.com/paper/1611.01734/) (2016) - [LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](https://mlsanity.com/paper/2010.01057/) (2020) - [Entity, Relation, and Event Extraction with Contextualized Span Representations](https://mlsanity.com/paper/1909.03546/) (2019) - [Named Entity Recognition with Bidirectional LSTM-CNNs](https://mlsanity.com/paper/1511.08308/) (2015) - [A Unified MRC Framework for Named Entity Recognition](https://mlsanity.com/paper/1910.11476/) (2019) - [Named Entity Recognition as Dependency Parsing](https://mlsanity.com/paper/2005.07150/) (2020) - [A Unified Generative Framework for Various NER Subtasks](https://mlsanity.com/paper/2106.01223/) (2021) - [Unified Named Entity Recognition as Word-Word Relation Classification](https://mlsanity.com/paper/2112.10070/) (2021) - [Chinese NER Using Lattice LSTM](https://mlsanity.com/paper/1805.02023/) (2018) - [An Effective Transition-based Model for Discontinuous NER](https://mlsanity.com/paper/2004.13454/) (2020) - [A Span-Based Model for Joint Overlapped and Discontinuous Named Entity Recognition](https://mlsanity.com/paper/2106.14373/) (2021) - [Neural Segmental Hypergraphs for Overlapping Mention Recognition](https://mlsanity.com/paper/1810.01817/) (2018) - [Low-Resource Name Tagging Learned with Weakly Labeled Data](https://mlsanity.com/paper/1908.09659/) (2019) - [Empower Sequence Labeling with Task-Aware Neural Language Model](https://mlsanity.com/paper/1709.04109/) (2017) - [Design Challenges and Misconceptions in Neural Sequence Labeling](https://mlsanity.com/paper/1806.04470/) (2018) - [Multilingual Language Processing From Bytes](https://mlsanity.com/paper/1512.00103/) (2015) - [Discontinuous Named Entity Recognition as Maximal Clique Discovery](https://mlsanity.com/paper/2106.00218/) (2021) - [Combining Spans into Entities: A Neural Two-Stage Approach for Recognizing Discontiguous Entities](https://mlsanity.com/paper/1909.00930/) (2019) ## Built on by (most cited first, 8 of 8) - [Reading Order Matters: Information Extraction from Visually-rich Documents by Token Path Prediction](https://mlsanity.com/paper/2310.11016/) (2023) - [A Survey on Arabic Named Entity Recognition: Past, Recent Advances, and Future Trends](https://mlsanity.com/paper/2302.03512/) (2023) - [TriG-NER: Triplet-Grid Framework for Discontinuous Named Entity Recognition](https://mlsanity.com/paper/2411.01839/) (2024) - [On Fusing ChatGPT and Ensemble Learning in Discon-tinuous Named Entity Recognition in Health Corpora](https://mlsanity.com/paper/2412.16976/) (2024) - [GapDNER: A Gap-Aware Grid Tagging Model for Discontinuous Named Entity Recognition](https://mlsanity.com/paper/2510.10927/) (2025) - [SEDA: A Self-Adapted Entity-Centric Data Augmentation for Boosting Gird-based Discontinuous NER Models](https://mlsanity.com/paper/2511.20143/) (2025) - [Code-MIE: A Code-style Model for Multimodal Information Extraction with Scene Graph and Entity Attribute Knowledge Enhancement](https://mlsanity.com/paper/2603.20781/) (2026) - [Analysing Lightweight Large Language Models for Biomedical Named Entity Recognition on Diverse Ouput Formats](https://mlsanity.com/paper/2604.25920/) (2026) ## Abstract (arXiv) So far, discontinuous named entity recognition (NER) has received increasing research attention and many related methods have surged such as hypergraph-based methods, span-based methods, and sequence-to-sequence (Seq2Seq) methods, etc. However, these methods more or less suffer from some problems such as decoding ambiguity and efficiency, which limit their performance. Recently, grid-tagging methods, which benefit from the flexible design of tagging systems and model architectures, have shown superiority to adapt for various information extraction tasks. In this paper, we follow the line of such methods and propose a competitive grid-tagging model for discontinuous NER. We call our model TOE because we incorporate two kinds of Tag-Oriented Enhancement mechanisms into a state-of-the-art (SOTA) grid-tagging model that casts the NER problem into word-word relationship prediction. First, we design a Tag Representation Embedding Module (TREM) to force our model to consider not only word-word relationships but also word-tag and tag-tag relationships. Concretely, we construct tag representations and embed them into TREM, so that TREM can treat tag and word representations as queries/keys/values and utilize self-attention to model their relationships. On the other hand, motivated by the Next-Neighboring-Word (NNW) and Tail-Head-Word (THW) tags in the SOTA model, we add two new symmetric tags, namely Previous-Neighboring-Word (PNW) and Head-Tail-Word (HTW), to model more fine-grained word-word relationships and alleviate error propagation from tag prediction. In the experiments of three benchmark datasets, namely CADEC, ShARe13 and ShARe14, our TOE model pushes the SOTA results by about 0.83%, 0.05% and 0.66% in F1, demonstrating its effectiveness.