mlsanityData to 7 Oct 2026

Cacheback: Speculative Decoding With Nothing But Cache

arXiv 2511.21699 ↗First posted 15 Nov 2025cs.CL cs.AIcited by 3

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

We present Cacheback Decoding, a training-free and model-agnostic speculative decoding method that exploits the locality in language to accelerate Large Language Model (LLM) inference. Cacheback leverages only Least Recently Used (LRU) cache tables of token n-grams to generate draft sequences. Cacheback achieves state-of-the-art performance among comparable methods despite its minimalist design, and its simplicity allows easy integration into existing systems. Cacheback also shows potential for fast adaptation to new domains.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 9 of its 14 references that are in the map. First posted on arXiv 15 Nov 2025.

Builds on

9 of 14 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024SAM Decoding: Speculative Decoding via Suffix Automaton16 Nov
  2. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decoding3 Feb
  3. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  4. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  5. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding15 Jan
  6. 2022Fast Inference from Transformers via Speculative Decoding30 Nov
  7. 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation30 Mar

Other references · 2

  1. 2024Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recyclingcited by 48
  2. 2023Lookahead: An Inference Acceleration Framework for Large Language Model with Lossless Generation Accuracycited by 37

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026AdaPLD: Adaptive Retrieval and Reuse for Efficient Model-Free Speculative Decodingcited by 1
  2. 2026WhiFlash: Accelerating Speculative Decoding with Token-Level Cross-Paradigm Routingcited by 1
  3. 2026Dynamic Delayed Tree Expansion For Improved Multi-Path Speculative Decodingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CARD: A Cache-Assisted Parallel Speculative Decoding Framework via Query-and-Correct Paradigm for Accelerating LLM Inference
  2. 2024Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding
  3. 2024SSSD: Simply-Scalable Speculative Decoding
  4. 2026Speculative Decoding with a Speculative Vocabulary
  5. 2025AdaSD: Adaptive Speculative Decoding for Efficient Language Model Inference
  6. 2026RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding
  7. 2026A Sparse Glimpse of the Whole: Train-Free Self-Speculative Decoding
  8. 2023Accelerating LLM Inference with Staged Speculative Decoding