mlsanityData to 7 Oct 2026

Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding

arXiv 2106.04970 ↗First posted 9 Jun 2021cs.CL cs.LGcited by 53

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

In this paper, we propose Shallow Aggressive Decoding (SAD) to improve the online inference efficiency of the Transformer for instantaneous Grammatical Error Correction (GEC). SAD optimizes the online inference efficiency for GEC by two innovations: 1) it aggressively decodes as many tokens as possible in parallel instead of always decoding only one token in each step to improve computational parallelism; 2) it uses a shallow decoder instead of the conventional Transformer architecture with balanced encoder-decoder depth to reduce the computational cost during inference. Experiments in both English and Chinese GEC benchmarks show that aggressive decoding could yield the same predictions as greedy decoding but with a significant speedup for online inference. Its combination with the shallow decoder could offer an even higher online inference speedup over the powerful Transformer baseline without quality loss. Not only does our approach allow a single model to achieve the state-of-the-art results in English GEC benchmarks: 66.4 F0.5 in the CoNLL-14 and 72.9 F0.5 in the BEA-19 test set with an almost 10x online inference speedup over the Transformer-big model, but also it is easily adapted to other languages. Our code is available at https://github.com/AutoTemp/Shallow-Aggressive-Decoding.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 22 of its 38 references that are in the map. First posted on arXiv 9 Jun 2021.

Builds on

22 of 38 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  7. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  9. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 318
  10. 2019Levenshtein Transformercited by 197
  11. 2020GECToR -- Grammatical Error Correction: Tag, Not Rewritecited by 177
  12. 2019Encode, Tag, Realize: High-Precision Text Editingcited by 99
10 more
  1. 2020Fully Non-autoregressive Neural Machine Translation: Tricks of the Tradecited by 99
  2. 2019Parallel Iterative Edit Models for Local Sequence Transductioncited by 81
  3. 2020Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correctioncited by 59
  4. 2020Seq2Edits: Sequence Transduction Using Span-level Edit Operationscited by 58
  5. 2019Corpora Generation for Grammatical Error Correctioncited by 53
  6. 2020Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correctioncited by 24
  7. 2018Reaching Human-level Performance in Automatic Grammatical Error Correction: An Empirical Studycited by 22
  8. 2019Improving Grammatical Error Correction with Machine Translation Pairscited by 17
  9. 2019Sequence-to-sequence Pre-training with Data Augmentation for Sentence Rewritingcited by 12
  10. 2021An Efficient Transformer Decoder with Compressed Sub-layerscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 9; 2023: 15; 2024: 10; 2025: 11; 2026: 7.

53 papers.

Sort:

By area:

  1. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  2. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 564
  3. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 362
  4. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 318
  5. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decodingcited by 266
  6. 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generationcited by 169
  7. 2023DistillSpec: Improving Speculative Decoding via Knowledge Distillationcited by 160
  8. 2023Accelerating Transformer Inference for Translation via Parallel Decodingcited by 123
  9. 2021CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generationcited by 81
  10. 2023Inference with Reference: Lossless Acceleration of Large Language Modelscited by 63
  11. 2025Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignmentcited by 59
  12. 2022Grammatical Error Correction: A Survey of the State of the Artcited by 43
38 more
  1. 2022EdiT5: Semi-Autoregressive Text-Editing with T5 Warm-Startcited by 35
  2. 2022SynGEC: Syntax-Enhanced Grammatical Error Correction with a Tailored GEC-Oriented Parsercited by 32
  3. 2025Accelerated Diffusion Models via Speculative Samplingcited by 30
  4. 2023UniTabE: A Universal Pretraining Protocol for Tabular Foundation Model in Data Sciencecited by 28
  5. 2022EdgeFormer: A Parameter-Efficient Transformer for On-Device Seq2seq Generationcited by 21
  6. 2022A Unified Strategy for Multilingual Grammatical Error Correction with Pre-trained Cross-Lingual Language Modelcited by 16
  7. 2023Disentangled Phonetic Representation for Chinese Spelling Correctioncited by 14
  8. 2024Speculative Decoding with CTC-based Draft Model for LLM Inference Accelerationcited by 14
  9. 2025L-MTP: Leap Multi-Token Prediction Beyond Adjacent Context for Large Language Modelscited by 14
  10. 2023Revisiting Event Argument Extraction: Can EAE Models Learn Better When Being Aware of Event Co-occurrences?cited by 10
  11. 2022Type-Driven Multi-Turn Corrections for Grammatical Error Correctioncited by 9
  12. 2023Non-autoregressive Text Editing with Copy-aware Latent Alignmentscited by 8
  13. 2023Improving Seq2Seq Grammatical Error Correction via Decoding Interventionscited by 8
  14. 2024S3D: A Simple and Cost-Effective Self-Speculative Decoding Scheme for Low-Memory GPUscited by 6
  15. 2024Improving Grammatical Error Correction via Contextual Data Augmentationcited by 6
  16. 2025Jakiro: Boosting Speculative Decoding with Decoupled Multi-Head via MoEcited by 6
  17. 2026Speculative Decoding with a Speculative Vocabularycited by 6
  18. 2026HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decodingcited by 5
  19. 2022CSynGEC: Incorporating Constituent-based Syntax for Grammatical Error Correction with a Tailored GEC-Oriented Parsercited by 4
  20. 2024Context-Aware Assistant Selection for Improved Inference Acceleration with Large Language Modelscited by 4
  21. 2025SpecVLM: Fast Speculative Decoding in Vision-Language Modelscited by 4
  22. 2025DynaSpec: Context-aware Dynamic Speculative Sampling for Large-Vocabulary Language Modelscited by 4
  23. 2025Steering Pretrained Drafters during Speculative Decodingcited by 4
  24. 2023Does Correction Remain A Problem For Large Language Models?cited by 3
  25. 2023GEC-DePenD: Non-Autoregressive Grammatical Error Correction with Decoupled Permutation and Decodingcited by 3
  26. 2023MixEdit: Revisiting Data Augmentation and Beyond for Grammatical Error Correctioncited by 2
  27. 2023Beyond Hard Samples: Robust and Effective Grammatical Error Correction with Cycle Self-Augmentingcited by 2
  28. 2024Decoding at the Speed of Thought: Harnessing Parallel Decoding of Lexical Units for LLMscited by 2
  29. 2025OverFill: Two-Stage Models for Efficient Language Model Decodingcited by 2
  30. 2023Bidirectional Transformer Reranker for Grammatical Error Correctioncited by 1
  31. 2023RedPenNet for Grammatical Error Correction: Outputs to Tokens, Attentions to Spanscited by 1
  32. 2023Grammatical Error Correction via Mixed-Grained Weighted Trainingcited by 1
  33. 2024Detection-Correction Structure via General Language Model for Grammatical Error Correctioncited by 1
  34. 2024S2D: Sorted Speculative Decoding For More Efficient Deployment of Nested Large Language Modelscited by 1
  35. 2025Tutorial Proposal: Speculative Decoding for Efficient LLM Inferencecited by 1
  36. 2025Cross-Attention Speculative Decodingcited by 0
  37. 2026Encode Errors: Representational Retrieval of In-Context Demonstrations for Multilingual Grammatical Error Correctioncited by 0
  38. 2026A Generalized Formalism of Auto-Regressive Decoding for Speech Processingcited by 0

50 most cited of 53 All 53 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Generation Meets Verification: Accelerating Large Language Model Inference with Smart Parallel Auto-Correct Decoding
  2. 2024ChatLang-8: An LLM-Based Synthetic Data Generation Framework for Grammatical Error Correction
  3. 2024Accelerating Large Language Model Inference with Self-Supervised Early Exits
  4. 2023Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding
  5. 2024Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding
  6. 2022Confident Adaptive Language Modeling
  7. 2023Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluation
  8. 2025Accelerating Large Language Model Inference via Early-Exiting Algorithms