# Hierarchical Contextualized Representation for Named Entity Recognition

arXiv: https://arxiv.org/abs/1911.02257 · HTML: https://mlsanity.com/paper/1911.02257/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Nov 2019.

## Built on by (most cited first, 30 of 32)

- [Template-Based Named Entity Recognition Using BART](https://mlsanity.com/paper/2106.01760/) (2021)
- [SpanNER: Named Entity Re-/Recognition as Span Prediction](https://mlsanity.com/paper/2106.00641/) (2021)
- [Packed Levitated Marker for Entity and Relation Extraction](https://mlsanity.com/paper/2109.06067/) (2021)
- [Pushing the Limits of ChatGPT on NLP Tasks](https://mlsanity.com/paper/2306.09719/) (2023)
- [Better Feature Integration for Named Entity Recognition](https://mlsanity.com/paper/2104.05316/) (2021)
- [Rethinking Negative Sampling for Handling Missing Entity Annotations](https://mlsanity.com/paper/2108.11607/) (2021)
- [Domain-Specific NER via Retrieving Correlated Samples](https://mlsanity.com/paper/2208.12995/) (2022)
- [Flat Multi-modal Interaction Transformer for Named Entity Recognition](https://mlsanity.com/paper/2208.11039/) (2022)
- [UniEX: An Effective and Efficient Framework for Unified Information Extraction via a Span-extractive Perspective](https://mlsanity.com/paper/2305.10306/) (2023)
- [Informed Named Entity Recognition Decoding for Generative Language Models](https://mlsanity.com/paper/2308.07791/) (2023)
- [Larger-Context Tagging: When and Why Does It Work?](https://mlsanity.com/paper/2104.04434/) (2021)
- [DualNER: A Dual-Teaching framework for Zero-shot Cross-lingual Named Entity Recognition](https://mlsanity.com/paper/2211.08104/) (2022)
- [ContrastNER: Contrastive-based Prompt Tuning for Few-shot NER](https://mlsanity.com/paper/2305.17951/) (2023)
- [Embedded Named Entity Recognition using Probing Classifiers](https://mlsanity.com/paper/2403.11747/) (2024)
- [Locally-Contextual Nonlinear CRFs for Sequence Labeling](https://mlsanity.com/paper/2103.16210/) (2021)
- [Named entity recognition architecture combining contextual and global features](https://mlsanity.com/paper/2112.08033/) (2021)
- [Boosting Entity Mention Detection for Targetted Twitter Streams with Global Contextual Embeddings](https://mlsanity.com/paper/2201.11885/) (2022)
- [Win-Win Cooperation: Bundling Sequence and Span Models for Named Entity Recognition](https://mlsanity.com/paper/2207.03300/) (2022)
- [Multi-Granularity Cross-Modality Representation Learning for Named Entity Recognition on Social Media](https://mlsanity.com/paper/2210.14163/) (2022)
- [Neural Sequence Segmentation as Determining the Leftmost Segments](https://mlsanity.com/paper/2104.07217/) (2021)
- [Plug-Tagger: A Pluggable Sequence Labeling Framework Using Language Models](https://mlsanity.com/paper/2110.07331/) (2021)
- [Identification of Fine-Grained Location Mentions in Crisis Tweets](https://mlsanity.com/paper/2111.06334/) (2021)
- [Improving Tagging Consistency and Entity Coverage for Chemical Identification in Full-text Articles](https://mlsanity.com/paper/2111.10584/) (2021)
- [SciEv: Finding Scientific Evidence Papers for Scientific News](https://mlsanity.com/paper/2205.00126/) (2022)
- [Cisco at AAAI-CAD21 shared task: Predicting Emphasis in Presentation Slides using Contextualized Embeddings](https://mlsanity.com/paper/2101.11422/) (2021)
- [Exploiting Global Contextual Information for Document-level Named Entity Recognition](https://mlsanity.com/paper/2106.00887/) (2021)
- [Nested and Balanced Entity Recognition using Multi-Task Learning](https://mlsanity.com/paper/2106.06216/) (2021)
- [KARL-Trans-NER: Knowledge Aware Representation Learning for Named Entity Recognition using Transformers](https://mlsanity.com/paper/2111.15436/) (2021)
- [Hero-Gang Neural Model For Named Entity Recognition](https://mlsanity.com/paper/2205.07177/) (2022)
- [GlobalNER: Incorporating Non-local Information into Named Entity Recognition](https://mlsanity.com/paper/2303.02915/) (2023)

## Abstract (arXiv)

Named entity recognition (NER) models are typically based on the architecture of Bi-directional LSTM (BiLSTM). The constraints of sequential nature and the modeling of single input prevent the full utilization of global information from larger scope, not only in the entire sentence, but also in the entire document (dataset). In this paper, we address these two deficiencies and propose a model augmented with hierarchical contextualized representation: sentence-level representation and document-level representation. In sentence-level, we take different contributions of words in a single sentence into consideration to enhance the sentence representation learned from an independent BiLSTM via label embedding attention mechanism. In document-level, the key-value memory network is adopted to record the document-aware information for each unique word which is sensitive to similarity of context information. Our two-level hierarchical contextualized representations are fused with each input token embedding and corresponding hidden state of BiLSTM, respectively. The experimental results on three benchmark NER datasets (CoNLL-2003 and Ontonotes 5.0 English datasets, CoNLL-2002 Spanish dataset) show that we establish new state-of-the-art results.