mlsanityData to 7 Oct 2026

CLLMs: Consistency Large Language Models

arXiv 2403.00835 ↗First posted 28 Feb 2024cs.CL cs.AIcited by 71

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Parallel decoding methods such as Jacobi decoding show promise for more efficient LLM inference as it breaks the sequential nature of the LLM decoding process and transforms it into parallelizable computation. However, in practice, it achieves little speedup compared to traditional autoregressive (AR) decoding, primarily because Jacobi decoding seldom accurately predicts more than one token in a single fixed-point iteration step. To address this, we develop a new approach aimed at realizing fast convergence from any state to the fixed point on a Jacobi trajectory. This is accomplished by refining the target LLM to consistently predict the fixed point given any state as input. Extensive experiments demonstrate the effectiveness of our method, showing 2.4× to 3.4× improvements in generation speed while preserving generation quality across both domain-specific and open-domain benchmarks.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 43 of its 51 references that are in the map. First posted on arXiv 28 Feb 2024.

Builds on

43 of 51 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decoding3 Feb
  2. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  3. 2023DistillSpec: Improving Speculative Decoding via Knowledge Distillation12 Oct
  4. 2023Accelerating Transformer Inference for Translation via Parallel Decoding17 May
  5. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  6. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 37

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  9. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  10. 2016Pointer Sentinel Mixture Modelscited by 2,992
  11. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  12. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
25 more
  1. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  2. 2023Consistency Modelscited by 1,778
  3. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  4. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  5. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  6. 2023PaLM 2 Technical Reportcited by 1,154
  7. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  8. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  9. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  10. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  11. 2023Large Language Modelscited by 774
  12. 2016Sequence-Level Knowledge Distillationcited by 761
  13. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  14. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 643
  15. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  16. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
  17. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  18. 2022Efficiently Scaling Transformer Inferencecited by 529
  19. 2023Improved Techniques for Training Consistency Modelscited by 430
  20. 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 408
  21. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 378
  22. 2022Structured Pruning Learns Compact and Accurate Modelscited by 167
  23. 2023MiniLLM: On-Policy Distillation of Large Language Modelscited by 129
  24. 2020Accelerating Feedforward Computation via Parallel Nonlinear Equation Solvingcited by 51
  25. 2020Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domainscited by 33

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 17; 2025: 32; 2026: 22.

71 papers.

Sort:

By area:

  1. 2025Large Language Diffusion Modelscited by 872
  2. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  3. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 362
  4. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
  5. 2024Scaling up Masked Diffusion Models on Textcited by 195
  6. 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 118
  7. 2025PD-VLA: Accelerating Vision-Language-Action Model Integrated with Action Chunking via Parallel Decodingcited by 88
  8. 2024SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengthscited by 73
  9. 2025Efficient Reasoning with Hidden Thinkingcited by 72
  10. 2024SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Accelerationcited by 69
  11. 2024Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decodingcited by 65
  12. 2024Parallelized Autoregressive Visual Generationcited by 58
38 more
  1. 2025DiffuSpec: Unlocking Diffusion Language Models for Speculative Decodingcited by 40
  2. 2024SAM Decoding: Speculative Decoding via Suffix Automatoncited by 38
  3. 2025CEED-VLA: Consistency Vision-Language-Action Model with Early-Exit Decodingcited by 38
  4. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillationcited by 34
  5. 2025Attention Is All You Need for KV Cache in Diffusion LLMscited by 32
  6. 2025Multiverse: Your Language Models Secretly Decide How to Parallelize and Merge Generationcited by 30
  7. 2025Speculative Decoding and Beyond: An In-Depth Survey of Techniquescited by 22
  8. 2024ZipAR: Parallel Auto-regressive Image Generation through Spatial Localitycited by 20
  9. 2025CDLM: Consistency Diffusion Language Models For Faster Samplingcited by 20
  10. 2025RobustFlow: Towards Robust Agentic Workflow Generationcited by 19
  11. 2024Continuous Speculative Decoding for Autoregressive Image Generationcited by 18
  12. 2024Distilled Decoding 1: One-step Sampling of Image Auto-regressive Models with Flow Matchingcited by 18
  13. 2024Mixture of Attentions For Speculative Decodingcited by 17
  14. 2025Accelerating Large Language Model Reasoning via Speculative Searchcited by 16
  15. 2024From Natural Language to SQL: Review of LLM-based Text-to-SQL Systemscited by 15
  16. 2025GRIFFIN: Effective Token Alignment for Faster Speculative Decodingcited by 15
  17. 2024Remote Timing Attacks on Efficient Language Model Inferencecited by 13
  18. 2025Scaling Speculative Decoding with Lookahead Reasoningcited by 13
  19. 2026JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Draftingcited by 12
  20. 2025Bridging Draft Policy Misalignment: Group Tree Optimization for Speculative Decodingcited by 9
  21. 2025PredGen: Accelerated Inference of Large Language Models through Input-Time Speculation for Real-Time Speech Interactioncited by 7
  22. 2025Speculative Jacobi-Denoising Decoding for Accelerating Autoregressive Text-to-image Generationcited by 7
  23. 2026xPress: Parallel Refinement for Diffusion Drafters in Speculative Decodingcited by 7
  24. 2025Think Before You Accept: Semantic Reflective Verification for Faster Speculative Decodingcited by 6
  25. 2025Asynchronous Fast-Slow Vision-Language-Action Policies for Whole-Body Robotic Manipulationcited by 6
  26. 2026LightningRL: Breaking the Accuracy-Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learningcited by 6
  27. 2026Introspective Diffusion Language Modelscited by 6
  28. 2025DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Modelscited by 5
  29. 2026Membership Inference Attacks Against Fine-tuned Diffusion Language Modelscited by 5
  30. 2024Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modelingcited by 4
  31. 2025TokenSwift: Lossless Acceleration of Ultra Long Sequence Generationcited by 4
  32. 2025Scaling Laws for Speculative Decodingcited by 4
  33. 2025Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunitiescited by 4
  34. 2025CAS-Spec: Cascade Adaptive Self-Speculative Decoding for On-the-Fly Lossless Inference Acceleration of LLMscited by 4
  35. 2024COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinementcited by 3
  36. 2025Masked Diffusion Models as Energy Minimizationcited by 3
  37. 2026Autoregressive Models Rival Diffusion Models at ANY-ORDER Generationcited by 3
  38. 2025PCM : Picard Consistency Model for Fast Parallel Sampling of Diffusion Modelscited by 2

50 most cited of 71 All 71 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Generation Meets Verification: Accelerating Large Language Model Inference with Smart Parallel Auto-Correct Decoding
  2. 2025Fast and Accurate Causal Parallel Decoding using Jacobi Forcing
  3. 2024Think Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decoding
  4. 2024Progressive Mixed-Precision Decoding for Efficient LLM Inference
  5. 2025ASPD: Unlocking Adaptive Serial-Parallel Decoding by Exploring Intrinsic Parallelism in LLMs
  6. 2025Accelerating Diffusion LLM Inference via Local Determinism Propagation
  7. 2026Faster LLM Inference via Sequential Monte Carlo
  8. 2024Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding