SPECTRA: Adaptive Execution of Speculative Decoding on a Runtime-Reconfigurable Tiled Architecture
On the map: Efficient LLM inference and serving → Speculative decoding
Abstract (arXiv)
LLM inference on edge devices is constrained by computational and memory resources, making efficient autoregressive decoding challenging. Speculative decoding alleviates this bottleneck by generating tokens with a smaller draft model and verifying multiple tokens in parallel with a batched target model pass. However, verification introduces a runtime-dependent intermediate regime between memory-bound general matrix-vector (GEMV) operations in decoding and compute-bound general matrix-matrix (GEMM) operations in prefill, as its arithmetic intensity varies with speculation length and acceptance rate. We present SPECTRA, a runtime-reconfigurable tiled architecture that sustains high utilization across the full speculative decoding pipeline. Within each tile, the compute engine switches between systolic execution for GEMMs and vector-lane execution for GEMVs. Across tiles, SPECTRA dynamically adapts computation parallelism by selecting tile count, kernel partitioning, and communication pattern. Both tile-level and system-level reconfiguration operate on a per-kernel basis, enabling efficient execution across these diverse regimes. Evaluated on a 20-tile FPGA prototype across the Pythia, SmolLM2, and GPT-2 families, SPECTRA achieves up to 2.09× speedup from tile-level reconfiguration and a further 1.25× gain from system-level adaptability over fixed designs.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 14 of its 28 references that are in the map. First posted on arXiv 21 Sep 2026.
Builds on
14 of 28 references.
Closest ancestors · 6
Same direction, newest first
- 2025SpecMamba: Accelerating Mamba Inference on FPGA with Speculative Decoding24 Sep
- 2025LP-Spec: Leveraging LPDDR PIM for Efficient LLM Mobile Speculative Inference with Architecture-Dataflow Co-Optimization10 Aug
- 2025Ghidorah: Fast LLM Inference on Edge with Speculative Decoding and Hetero-Core Parallelism29 May
- 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification16 May
- 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
- 2022Fast Inference from Transformers via Speculative Decoding30 Nov
Other references · 8
- 2017Attention Is All You Needcited by 60,361
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
- 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
- 2019The Cost of Application-Class Processing: Energy and Performance Analysis of a Linux-ready 1.7GHz 64bit RISC-V Core in 22nm FDSOI Technologycited by 17
- 2020Agile SoC Development with Open ESPcited by 5
- 2024TATAA: Programmable Mixed-Precision Transformer Acceleration with a Transformable Arithmetic Architecturecited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Compiler-Assisted Speculative Sampling for Accelerated LLM Inference on Heterogeneous Edge Devices
- 2025SLED: A Speculative LLM Decoding Framework for Efficient Edge Serving
- 2026MemSpec: Memory-Aware Runtime for Adaptive Draft Scheduling in Speculative Decoding on Edge Devices
- 2026CATS: Cascaded Adaptive Tree Speculation for Memory-Limited LLM Inference Acceleration
- 2025Mirror Speculative Decoding: Breaking the Serial Barrier in LLM Inference
- 2026BitNest: Bit-Nested Speculative Decoding for Memory-Efficient LLM Inference Acceleration
- 2025Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Serving
- 2025CARD: A Cache-Assisted Parallel Speculative Decoding Framework via Query-and-Correct Paradigm for Accelerating LLM Inference