# TENER: Adapting Transformer Encoder for Named Entity Recognition

arXiv: https://arxiv.org/abs/1911.04474 · HTML: https://mlsanity.com/paper/1911.04474/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Nov 2019.

## Built on by (most cited first, 30 of 69)

- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [A Unified Generative Framework for Various NER Subtasks](https://mlsanity.com/paper/2106.01223/) (2021)
- [MetaFormer Baselines for Vision](https://mlsanity.com/paper/2210.13452/) (2022)
- [Position Information in Transformers: An Overview](https://mlsanity.com/paper/2102.11090/) (2021)
- [Mixed Transformer U-Net For Medical Image Segmentation](https://mlsanity.com/paper/2111.04734/) (2021)
- [Unified Named Entity Recognition as Word-Word Relation Classification](https://mlsanity.com/paper/2112.10070/) (2021)
- [A Temporal Kolmogorov-Arnold Transformer for Time Series Forecasting](https://mlsanity.com/paper/2406.02486/) (2024)
- [A Partition Filter Network for Joint Entity and Relation Extraction](https://mlsanity.com/paper/2108.12202/) (2021)
- [Dissecting Transformer Length Extrapolation via the Lens of Receptive Field Analysis](https://mlsanity.com/paper/2212.10356/) (2022)
- [Accelerating BERT Inference for Sequence Labeling via Early-Exit](https://mlsanity.com/paper/2105.13878/) (2021)
- [Pushing the Limits of ChatGPT on NLP Tasks](https://mlsanity.com/paper/2306.09719/) (2023)
- [NBIAS: A Natural Language Processing Framework for Bias Identification in Text](https://mlsanity.com/paper/2308.01681/) (2023)
- [When Large Language Models Meet Evolutionary Algorithms: Potential Enhancements and Challenges](https://mlsanity.com/paper/2401.10510/) (2024)
- [MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspective](https://mlsanity.com/paper/2204.04391/) (2022)
- [Simplified DOM Trees for Transferable Attribute Extraction from the Web](https://mlsanity.com/paper/2101.02415/) (2021)
- [TransLIST: A Transformer-Based Linguistically Informed Sanskrit Tokenizer](https://mlsanity.com/paper/2210.11753/) (2022)
- [Understanding the Failure of Batch Normalization for Transformers in NLP](https://mlsanity.com/paper/2210.05153/) (2022)
- [PiSLTRc: Position-informed Sign Language Transformer with Content-aware Convolution](https://mlsanity.com/paper/2107.12600/) (2021)
- [TE-ESN: Time Encoding Echo State Network for Prediction Based on Irregularly Sampled Time Series Data](https://mlsanity.com/paper/2105.00412/) (2021)
- [O-ViT: Orthogonal Vision Transformer](https://mlsanity.com/paper/2201.12133/) (2022)
- [Token-level Sequence Labeling for Spoken Language Understanding using Compositional End-to-End Models](https://mlsanity.com/paper/2210.15734/) (2022)
- [PUnifiedNER: A Prompting-based Unified NER System for Diverse Datasets](https://mlsanity.com/paper/2211.14838/) (2022)
- [End-to-End Dense Video Grounding via Parallel Regression](https://mlsanity.com/paper/2109.11265/) (2021)
- [Less is More: Pre-train a Strong Text Encoder for Dense Retrieval Using a Weak Decoder](https://mlsanity.com/paper/2102.09206/) (2021)
- [Multi-Scale Local-Temporal Similarity Fusion for Continuous Sign Language Recognition](https://mlsanity.com/paper/2107.12762/) (2021)
- [An Embarrassingly Easy but Strong Baseline for Nested Named Entity Recognition](https://mlsanity.com/paper/2208.04534/) (2022)
- [MAWSEO: Adversarial Wiki Search Poisoning for Illicit Online Promotion](https://mlsanity.com/paper/2304.11300/) (2023)
- [E-NER: Evidential Deep Learning for Trustworthy Named Entity Recognition](https://mlsanity.com/paper/2305.17854/) (2023)
- [Speech2Slot: An End-to-End Knowledge-based Slot Filling from Speech](https://mlsanity.com/paper/2105.04719/) (2021)
- [Hallucinated Span Detection with Multi-View Attention Features](https://mlsanity.com/paper/2504.04335/) (2025)

## Abstract (arXiv)

The Bidirectional long short-term memory networks (BiLSTM) have been widely used as an encoder in models solving the named entity recognition (NER) task. Recently, the Transformer is broadly adopted in various Natural Language Processing (NLP) tasks owing to its parallelism and advantageous performance. Nevertheless, the performance of the Transformer in NER is not as good as it is in other NLP tasks. In this paper, we propose TENER, a NER architecture adopting adapted Transformer Encoder to model the character-level features and word-level features. By incorporating the direction and relative distance aware attention and the un-scaled attention, we prove the Transformer-like encoder is just as effective for NER as other NLP tasks.