mlsanityData to 7 Oct 2026

PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognition

arXiv 2402.04838 ↗First posted 7 Feb 2024cs.CL cs.AIcited by 6

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

In this study, we aim to reduce generation latency for Named Entity Recognition (NER) with Large Language Models (LLMs). The main cause of high latency in LLMs is the sequential decoding process, which autoregressively generates all labels and mentions for NER, significantly increase the sequence length. To this end, we introduce Parallel Decoding in LLM for NE} (PaDeLLM-NER), a approach that integrates seamlessly into existing generative model frameworks without necessitating additional modules or architectural modifications. PaDeLLM-NER allows for the simultaneous decoding of all mentions, thereby reducing generation latency. Experiments reveal that PaDeLLM-NER significantly increases inference speed that is 1.76 to 10.22 times faster than the autoregressive approach for both English and Chinese. Simultaneously it maintains the quality of predictions as evidenced by the performance that is on par with the state-of-the-art across various datasets.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 56 of its 82 references that are in the map. First posted on arXiv 7 Feb 2024.

Builds on

56 of 82 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  2. 2023Accelerating Transformer Inference for Translation via Parallel Decoding17 May
  3. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  4. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 52

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  9. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  10. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  11. 2023Qwen Technical Reportcited by 3,273
  12. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
38 more
  1. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  2. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  3. 2023LIMA: Less Is More for Alignmentcited by 1,080
  4. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  5. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  6. 2022Confident Adaptive Language Modelingcited by 297
  7. 2021ICDAR2019 Competition on Scanned Receipt OCR and Information Extractioncited by 254
  8. 2020SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruningcited by 228
  9. 2019Depth-Adaptive Transformercited by 227
  10. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 221
  11. 2022Unified Structure Generation for Universal Information Extractioncited by 220
  12. 2021Structured Prediction as Translation between Augmented Natural Languagescited by 183
  13. 2023Large Language Models for Generative Information Extraction: A Surveycited by 183
  14. 2023InstructUIE: Multi-task Instruction Tuning for Unified Information Extractioncited by 167
  15. 2023GPT-NER: Named Entity Recognition via Large Language Modelscited by 158
  16. 2023GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extractioncited by 116
  17. 2020CrossNER: Evaluating Cross-Domain Named Entity Recognitioncited by 102
  18. 2021Unified Named Entity Recognition as Word-Word Relation Classificationcited by 89
  19. 2023Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Taskscited by 84
  20. 2023GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformercited by 84
  21. 2018Chinese NER Using Lattice LSTMcited by 81
  22. 2023UniDoc: A Universal Large Multimodal Model for Simultaneous Text Detection, Recognition, Spotting and Understandingcited by 74
  23. 2023UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognitioncited by 70
  24. 2023Inference with Reference: Lossless Acceleration of Large Language Modelscited by 63
  25. 2023Empirical Study of Zero-Shot NER with ChatGPTcited by 60
  26. 2022DeepStruct: Pretraining of Language Models for Structure Predictioncited by 45
  27. 2023Learning Transformer Programscited by 40
  28. 2023PromptNER: Prompting For Named Entity Recognitioncited by 35
  29. 2021Lexicon Enhanced Chinese Sequence Labeling Using BERT Adaptercited by 34
  30. 2022A Rationale-Centric Framework for Human-in-the-loop Machine Learningcited by 27
  31. 2023Copy Is All You Needcited by 25
  32. 2022Optimizing Bi-Encoder for Named Entity Recognition via Contrastive Learningcited by 24
  33. 2019Simplify the Usage of Lexicon in Chinese NERcited by 23
  34. 2022What Makes Pre-trained Language Models Better Zero-shot Learners?cited by 23
  35. 2023Unified Low-Resource Sequence Labeling by Sample-Aware Dynamic Sparse Finetuningcited by 21
  36. 2022Domain-Specific NER via Retrieving Correlated Samplescited by 11
  37. 2023Deeply Coupled Cross-Modal Prompt Learningcited by 11
  38. 2022PUnifiedNER: A Prompting-based Unified NER System for Diverse Datasetscited by 10

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 3.

6 papers.

  1. 2024A Bounding Box is Worth One Token: Interleaving Layout and Text in a Large Language Model for Document Understandingcited by 57
  2. 2026Dolphin-v2: Universal Document Parsing via Scalable Anchor Promptingcited by 20
  3. 2025Accelerating Large Language Model Reasoning via Speculative Searchcited by 16
  4. 2024LTNER: Large Language Model Tagging for Named Entity Recognition with Contextualized Entity Markingcited by 7
  5. 2026Up to 36x Speedup: Mask-based Parallel Inference Paradigm for Key Information Extraction in MLLMscited by 0
  6. 2026H2: A Dual Hybrid Semantic Data Lake Architecture for Medical Data Harmonization with Human-In-the-Loop verified, LLM Driven Metadata Annotation Systemcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Self-Improving for Zero-Shot Named Entity Recognition with Large Language Models
  2. 2024Rethinking Negative Instances for Generative Named Entity Recognition
  3. 2022kNN-NER: Named Entity Recognition with Nearest Neighbor Search
  4. 2021Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training
  5. 2025CodeNER: Code Prompting for Named Entity Recognition
  6. 2024Improving Few-Shot Cross-Domain Named Entity Recognition by Instruction Tuning a Word-Embedding based Retrieval Augmented Large Language Model
  7. 2024Decoding at the Speed of Thought: Harnessing Parallel Decoding of Lexical Units for LLMs
  8. 2024ReverseNER: A Self-Generated Example-Driven Framework for Zero-Shot Named Entity Recognition with Large Language Models