mlsanityData to 7 Oct 2026

Generation Meets Verification: Accelerating Large Language Model Inference with Smart Parallel Auto-Correct Decoding

arXiv 2402.11809 ↗First posted 19 Feb 2024cs.CL cs.AI cs.LGcited by 22

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

This research aims to accelerate the inference speed of large language models (LLMs) with billions of parameters. We propose Smart Parallel Auto-Correct dEcoding (SPACE), an innovative approach designed for achieving lossless acceleration of LLMs. By integrating semi-autoregressive inference and speculative decoding capabilities, SPACE uniquely enables autoregressive LLMs to parallelize token generation and verification. This is realized through a specialized semi-autoregressive supervised fine-tuning process that equips existing LLMs with the ability to simultaneously predict multiple tokens. Additionally, an auto-correct decoding algorithm facilitates the simultaneous generation and verification of token sequences within a single model invocation. Through extensive experiments on a range of LLMs, SPACE has demonstrated inference speedup ranging from 2.7x-4.0x on HumanEval-X while maintaining output quality.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 24 of its 48 references that are in the map. First posted on arXiv 19 Feb 2024.

Builds on

24 of 48 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decoding3 Feb
  2. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  3. 2023Accelerating LLM Inference with Staged Speculative Decoding8 Aug
  4. 2023Speculative Decoding with Big Little Decoder15 Feb
  5. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  6. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  6. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  7. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  8. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  9. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  10. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  11. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  12. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
6 more
  1. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  2. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 318
  3. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 294
  4. 2023Platypus: Quick, Cheap, and Powerful Refinement of LLMscited by 150
  5. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  6. 2018Semi-Autoregressive Neural Machine Translationcited by 48

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 10; 2026: 5.

22 papers.

Sort:

By area:

  1. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 362
  2. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 318
  3. 2024SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Accelerationcited by 69
  4. 2025Speculative Decoding and Beyond: An In-Depth Survey of Techniquescited by 22
  5. 2024ParallelSpec: Parallel Drafter for Efficient Speculative Decodingcited by 20
  6. 2024Continuous Speculative Decoding for Autoregressive Image Generationcited by 18
  7. 2025Overconfidence in LLM-as-a-Judge: Diagnosis and Confidence-Driven Solutioncited by 17
  8. 2025Fast Large Language Model Collaborative Decoding via Speculationcited by 12
  9. 2025SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inferencecited by 12
  10. 2025ASPD: Unlocking Adaptive Serial-Parallel Decoding by Exploring Intrinsic Parallelism in LLMscited by 11
  11. 2024Clover: Regressive Lightweight Speculative Decoding with Sequential Knowledgecited by 10
  12. 2024A Comprehensive Survey of Accelerated Generation Techniques in Large Language Modelscited by 5
10 more
  1. 2026SLMJury: Can Small Language Models Judge as Well as Large Ones?cited by 3
  2. 2025HeteroSpec: Leveraging Contextual Heterogeneity for Efficient Speculative Decodingcited by 2
  3. 2026PACER: Blockwise Pre-verification for Speculative Decoding with Adaptive Lengthcited by 2
  4. 2026BubbleSpec: Turning Long-Tail Bubbles into Speculative Rollout Drafts for Synchronous Reinforcement Learningcited by 2
  5. 2024S2D: Sorted Speculative Decoding For More Efficient Deployment of Nested Large Language Modelscited by 1
  6. 2025Speculate Deep and Accurate: Lossless and Training-Free Acceleration for Offloaded LLMs via Substitute Speculative Decodingcited by 1
  7. 2025Mirror Speculative Decoding: Breaking the Serial Barrier in LLM Inferencecited by 1
  8. 2025Polybasic Speculative Decoding Through a Theoretical Perspectivecited by 1
  9. 2026SpecBound: Adaptive Bounded Self-Speculation with Layer-wise Confidence Calibrationcited by 1
  10. 2026SDFP: Speculative Decoding with FIT-Pruned Models for Training-Free and Plug-and-Play LLM Accelerationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding
  2. 2024Minions: Accelerating Large Language Model Inference with Aggregated Speculative Execution
  3. 2024Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding
  4. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding
  5. 2024Accelerating Large Language Model Inference with Self-Supervised Early Exits
  6. 2024Falcon: Faster and Parallel Inference of Large Language Models through Enhanced Semi-Autoregressive Drafting and Custom-Designed Decoding Tree
  7. 2024Optimized Multi-Token Joint Decoding with Auxiliary Model for LLM Inference
  8. 2026Faster LLM Inference via Sequential Monte Carlo