mlsanityData to 7 Oct 2026

31.1 A 14.08-to-135.69Token/s ReRAM-on-Logic Stacked Outlier-Free Large-Language-Model Accelerator with Block-Clustered Weight-Compression and Adaptive Parallel-Speculative-Decoding

arXiv 2605.09375 ↗First posted 10 May 2026cs.ARcited by 0

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

This work presents a 55nm speculative decoding-based LLM accelerator with bumping-based face-to-face ReRAM-on-logic stacking technology. It features a local rotation unit for outlier-free low-bit quantization, a stacking-aware PNM architecture co-designed with blockwise vector quantization to reduce weight EMA overheads, and an adaptive parallel speculative decoding scheme with an out-of-order scheduler for high resource and bandwidth utilization. Our chip achieves 14.08-to-135.69token/s and 4.46-to-7.17x speedup over vanilla speculative decoding.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 14 of its 20 references that are in the map. First posted on arXiv 10 May 2026.

Builds on

14 of 20 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024PEARL: Parallel Speculative Decoding with Adaptive Draft Length13 Aug
  2. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  3. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 11

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  4. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
  5. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  6. 2024GPTVQ: The Blessing of Dimensionality for LLM Quantizationcited by 78
  7. 2024VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Modelscited by 53
  8. 2024MaskLLM: Learnable Semi-Structured Sparsity for Large Language Modelscited by 49
  9. 2024RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantizationcited by 13
  10. 2025A 28nm 0.22μJ/token memory-compute-intensity-aware CNN-Transformer accelerator with hybrid-attention-based layer-fusion and cascaded pruning for semantic-segmentationcited by 3
  11. 2024MVQ:Towards Efficient DNN Compression and Acceleration with Masked Vector Quantizationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025From Quarter to All: Accelerating Speculative LLM Decoding via Floating-Point Exponent Remapping and Parameter Sharing
  2. 2026SPECTRA: Adaptive Execution of Speculative Decoding on a Runtime-Reconfigurable Tiled Architecture
  3. 2024AMUSD: Asynchronous Multi-Device Speculative Decoding for LLM Acceleration
  4. 2026CATS: Cascaded Adaptive Tree Speculation for Memory-Limited LLM Inference Acceleration
  5. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
  6. 2025Scaling LLM Speculative Decoding: Non-Autoregressive Forecasting in Large-Batch Scenarios
  7. 2025Helix Parallelism: Rethinking Sharding Strategies for Interactive Multi-Million-Token LLM Decoding
  8. 2026BitNest: Bit-Nested Speculative Decoding for Memory-Efficient LLM Inference Acceleration