mlsanityData to 14 Oct 2026

RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding

arXiv 2604.14885 ↗First posted 16 Apr 2026cs.CL cs.AIcited by 4

On the map: Retrieval-augmented generation → Generation and context integration

also in 1 more
Abstract (arXiv)

Autoregressive decoding in Large Language Models (LLMs) generates one token per step, causing high inference latency. Speculative decoding (SD) mitigates this through a guess-and-verify strategy, but existing training-free variants face trade-offs: retrieval-based drafts break when no exact match exists, while logits-based drafts lack structural guidance. We propose RACER (Retrieval-Augmented Contextual Rapid Speculative Decoding), a lightweight and training-free method that integrates retrieved exact patterns with logit-driven future cues. This unification supplies both reliable anchors and flexible extrapolation, yielding richer speculative drafts. Experiments on Spec-Bench, HumanEval, and MGSM-ZH demonstrate that RACER consistently accelerates inference, achieving more than 2× speedup over autoregressive decoding, and outperforms prior training-free methods, offering a scalable, plug-and-play solution for efficient LLM decoding. Our source code is available at hrefhttps://github.com/hkr04/RACERhttps://github.com/hkr04/RACER.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 18 of its 31 references that are in the map. First posted on arXiv 16 Apr 2026.

Builds on

18 of 31 references.

References · 18

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  5. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  6. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  7. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  8. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 578
  9. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 370
  10. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 360
  11. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 331
  12. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decodingcited by 269
6 more
  1. 2024Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumptioncited by 97
  2. 2024PEARL: Parallel Speculative Decoding with Adaptive Draft Lengthcited by 85
  3. 2024Learning Harmonized Representations for Speculative Samplingcited by 72
  4. 2024Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recyclingcited by 47
  5. 2025LogitSpec: Accelerating Retrieval-based Speculative Decoding via Next Next Token Speculationcited by 14
  6. 2025Scaling LLM Speculative Decoding: Non-Autoregressive Forecasting in Large-Batch Scenarioscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026TIGER: Text-Conditioned Visual Gated Routing with Acceptance Alignment for Multimodal Speculative Decodingcited by 2
  2. 2026SPECTRE: Hybrid Ordinary-Parallel Speculative Serving for Resource-Efficient LLM Inferencecited by 0
  3. 2026Oilbird: Training-Free Speculative Decoding with Keys the Verifier Already Computescited by 0
  4. 2026ECHO: Early-layer Collaborative Hierarchical Orchestration with Bonus Logits in Speculative Decodingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding
  2. 2024SSSD: Simply-Scalable Speculative Decoding
  3. 2024A Theoretical Perspective for Speculative Decoding Algorithm
  4. 2026Speculative Speculative Decoding
  5. 2026A Practical Investigation of Training-free Relaxed Speculative Decoding
  6. 2026SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding
  7. 2026LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter
  8. 2025Draft, Verify, and Improve: Toward Training-Aware Speculative Decoding