mlsanityData to 14 Oct 2026

Speculative Decoding for Verilog: Speed and Quality, All in One

arXiv 2503.14153 ↗First posted 18 Mar 2025cs.LG cs.AR cs.CLcited by 2

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

The rapid advancement of large language models (LLMs) has revolutionized code generation tasks across various programming languages. However, the unique characteristics of programming languages, particularly those like Verilog with specific syntax and lower representation in training datasets, pose significant challenges for conventional tokenization and decoding approaches. In this paper, we introduce a novel application of speculative decoding for Verilog code generation, showing that it can improve both inference speed and output quality, effectively achieving speed and quality all in one. Unlike standard LLM tokenization schemes, which often fragment meaningful code structures, our approach aligns decoding stops with syntactically significant tokens, making it easier for models to learn the token distribution. This refinement addresses inherent tokenization issues and enhances the model's ability to capture Verilog's logical constructs more effectively. Our experimental results show that our method achieves up to a 5.05x speedup in Verilog code generation and increases pass@10 functional accuracy on RTLLM by up to 17.19% compared to conventional training strategies. These findings highlight speculative decoding as a promising approach to bridge the quality gap in code generation for specialized programming languages.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 29 of its 38 references that are in the map. First posted on arXiv 18 Mar 2025.

Builds on

29 of 38 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  2. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification16 May
  3. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  4. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 25

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  3. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  4. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  5. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  6. 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 361
  7. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 333
  8. 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 331
  9. 2023VerilogEval: Evaluating Large Language Models for Verilog Code Generationcited by 252
  10. 2023VeriGen: A Large Language Model for Verilog Code Generationcited by 200
  11. 2023RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Modelcited by 148
  12. 2023Chip-Chat: Challenges and Opportunities in Conversational Hardware Designcited by 140
13 more
  1. 2022Benchmarking Large Language Models for Automated Verilog RTL Code Generationcited by 115
  2. 2023ChipGPT: How far are we from natural language hardware designcited by 105
  3. 2017Abstract Syntax Networks for Code Generation and Semantic Parsingcited by 92
  4. 2023RTLCoder: Outperforming GPT-3.5 in Design RTL Generation with Our Open-Source Dataset and Lightweight Solutioncited by 71
  5. 2021SynCoBERT: Syntax-Guided Multi-Modal Contrastive Pre-Training for Code Representationcited by 63
  6. 2019TreeGen: A Tree-Based Transformer Architecture for Code Generationcited by 58
  7. 2024OriGen:Enhancing RTL Code Generation with Code-to-Code Augmentation and Self-Reflectioncited by 56
  8. 2020DAVE: Deriving Automatically Verilog from Englishcited by 48
  9. 2024MG-Verilog: Multi-grained Dataset Towards Enhanced LLM-assisted Verilog Generationcited by 46
  10. 2021TreeBERT: A Tree-Based Pre-Trained Model for Programming Languagecited by 41
  11. 2024Data is all you need: Finetuning LLMs for Chip Design via an Automated design-data augmentation frameworkcited by 36
  12. 2018A Grammar-Based Structural CNN Decoder for Code Generationcited by 29
  13. 2017PrivMin: Differentially Private MinHash for Jaccard Similarity Computationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Large Language Model for Verilog Code Generation: Literature Review and the Road Aheadcited by 8
  2. 2026CURE: Local Uncertainty Repair for Block-Parallel Speculative Decodingcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Speculative Decoding: Performance or Illusion?
  2. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding
  3. 2024Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding
  4. 2024Recursive Speculative Decoding: Accelerating LLM Inference via Sampling Without Replacement
  5. 2026SpecForge: A Flexible and Efficient Open-Source Training Framework for Speculative Decoding
  6. 2026SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding
  7. 2024Decoding Speculative Decoding
  8. 2025Tutorial Proposal: Speculative Decoding for Efficient LLM Inference