# Fast and Accurate Entity Recognition with Iterated Dilated Convolutions

arXiv: https://arxiv.org/abs/1702.02098 · HTML: https://mlsanity.com/paper/1702.02098/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Feb 2017.

## Built on by (most cited first, 30 of 40)

- [Mega: Moving Average Equipped Gated Attention](https://mlsanity.com/paper/2209.10655/) (2022)
- [Template-Based Named Entity Recognition Using BART](https://mlsanity.com/paper/2106.01760/) (2021)
- [Unified Named Entity Recognition as Word-Word Relation Classification](https://mlsanity.com/paper/2112.10070/) (2021)
- [Conditional probing: measuring usable information beyond a baseline](https://mlsanity.com/paper/2109.09234/) (2021)
- [Text-to-Table: A New Way of Information Extraction](https://mlsanity.com/paper/2109.02707/) (2021)
- [InstructionNER: A Multi-Task Instruction-Based Generative Framework for Few-shot NER](https://mlsanity.com/paper/2203.03903/) (2022)
- [LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting](https://mlsanity.com/paper/2109.00720/) (2021)
- [Better Feature Integration for Named Entity Recognition](https://mlsanity.com/paper/2104.05316/) (2021)
- [Global Pointer: Novel Efficient Span-based Approach for Named Entity Recognition](https://mlsanity.com/paper/2208.03054/) (2022)
- [Recurrent Attention Networks for Long-text Modeling](https://mlsanity.com/paper/2306.06843/) (2023)
- [What do we Really Know about State of the Art NER?](https://mlsanity.com/paper/2205.00034/) (2022)
- [On the Evolution of Knowledge Graphs: A Survey and Perspective](https://mlsanity.com/paper/2310.04835/) (2023)
- [Kuaipedia: a Large-scale Multi-modal Short-video Encyclopedia](https://mlsanity.com/paper/2211.00732/) (2022)
- [ToNER: Type-oriented Named Entity Recognition with Generative Language Model](https://mlsanity.com/paper/2404.09145/) (2024)
- [Bidirectional End-to-End Learning of Retriever-Reader Paradigm for Entity Linking](https://mlsanity.com/paper/2306.12245/) (2023)
- [Neural Contract Element Extraction Revisited: Letters from Sesame Street](https://mlsanity.com/paper/2101.04355/) (2021)
- [A novel knowledge graph development for industry design: A case study on indirect coal liquefaction process](https://mlsanity.com/paper/2111.13854/) (2021)
- [Cross-Lingual Transfer Robustness to Lower-Resource Languages on Adversarial Datasets](https://mlsanity.com/paper/2403.20056/) (2024)
- [Larger-Context Tagging: When and Why Does It Work?](https://mlsanity.com/paper/2104.04434/) (2021)
- [DyLex: Incorporating Dynamic Lexicons into BERT for Sequence Labeling](https://mlsanity.com/paper/2109.08818/) (2021)
- [Multi-task Transformer with Relation-attention and Type-attention for Named Entity Recognition](https://mlsanity.com/paper/2303.10870/) (2023)
- [Enhanced Language Representation with Label Knowledge for Span Extraction](https://mlsanity.com/paper/2111.00884/) (2021)
- [Korean Named Entity Recognition Based on Language-Specific Features](https://mlsanity.com/paper/2305.06330/) (2023)
- [COMBO: State-of-the-Art Morphosyntactic Analysis](https://mlsanity.com/paper/2109.05361/) (2021)
- [Distantly supervised end-to-end medical entity extraction from electronic health records with human-level quality](https://mlsanity.com/paper/2201.10463/) (2022)
- [Win-Win Cooperation: Bundling Sequence and Span Models for Named Entity Recognition](https://mlsanity.com/paper/2207.03300/) (2022)
- [Multi-Granularity Cross-Modality Representation Learning for Named Entity Recognition on Social Media](https://mlsanity.com/paper/2210.14163/) (2022)
- [Span-Oriented Information Extraction -- A Unifying Perspective on Information Extraction](https://mlsanity.com/paper/2403.15453/) (2024)
- [FreeTransfer-X: Safe and Label-Free Cross-Lingual Transfer from Off-the-Shelf Models](https://mlsanity.com/paper/2206.06586/) (2022)
- [IXA/Cogcomp at SemEval-2023 Task 2: Context-enriched Multilingual Named Entity Recognition using Knowledge Bases](https://mlsanity.com/paper/2304.10637/) (2023)

## Abstract (arXiv)

Today when many practitioners run basic NLP on the entire web and large-volume traffic, faster methods are paramount to saving time and energy costs. Recent advances in GPU hardware have led to the emergence of bi-directional LSTMs as a standard method for obtaining per-token vector representations serving as input to labeling tasks such as NER (often followed by prediction in a linear-chain CRF). Though expressive and accurate, these models fail to fully exploit GPU parallelism, limiting their computational efficiency. This paper proposes a faster alternative to Bi-LSTMs for NER: Iterated Dilated Convolutional Neural Networks (ID-CNNs), which have better capacity than traditional CNNs for large context and structured prediction. Unlike LSTMs whose sequential processing on sentences of length N requires O(N) time even in the face of parallelism, ID-CNNs permit fixed-depth convolutions to run in parallel across entire documents. We describe a distinct combination of network structure, parameter sharing and training procedures that enable dramatic 14-20x test-time speedups while retaining accuracy comparable to the Bi-LSTM-CRF. Moreover, ID-CNNs trained to aggregate context from the entire document are even more accurate while maintaining 8x faster test time speeds.