mlsanityData to 14 Oct 2026

AsymSpec: Efficient Cloud-Edge Speculative Decoding over Asymmetric Networks

arXiv 2608.04974 ↗First posted 5 Aug 2026cs.DCcited by 0

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

Cloud-edge speculative decoding places a lightweight draft model at an edge gateway and a higher-quality target model in the cloud, but inserts communication into every speculative block. Under a constrained uplink, candidate messages may queue while the verifier is idle. Stop-and-wait scheduling leaves edge compute underutilized; optimistic same-request runahead can waste work when a rejection or an unexpected bonus token invalidates dependent drafts. We present AsymSpec, which addresses uplink-gated verification and invalid dependent work with two corresponding mechanisms. Its asymmetric verification protocol keeps the common-path acceptance upload compact and moves richer, rejection-only correction information to the downlink. A total-variation (TV) certificate for the residual distribution determines whether a small target top-K response suffices; if not, the protocol progressively escalates through proposal-based exact recovery before falling back to the full distribution. Its confirmed-prefix pipeline exposes only independent, valid requests to the edge scheduler and lets the cloud re-batch arrived blocks, hiding verification waits when another confirmed-prefix request is ready without using same-request runahead. Across three draft-target pairs, two workloads, and three asymmetric network profiles, our end-to-end evaluation shows that AsymSpec delivers 2.82-28.03× the output-token throughput of the strongest baseline.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 30 of its 37 references that are in the map. First posted on arXiv 5 Aug 2026.

Builds on

30 of 37 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Speculation at a Distance: Where Edge-Cloud Speculative Decoding Actually Pays Off23 Jun
  2. 2026PipeSD: An Efficient Cloud-Edge Collaborative Pipeline Inference Framework with Speculative Decoding13 May
  3. 2025Conformal Sparsification for Bandwidth-Efficient Edge-Cloud Speculative Decoding11 Oct

Other references · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2025Qwen3 Technical Reportcited by 8,820
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  8. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  9. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  10. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  11. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  12. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
15 more
  1. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  2. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  3. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 526
  4. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 365
  5. 2024Sequoia: Scalable, Robust, and Hardware-aware Speculative Decodingcited by 103
  6. 2024PEARL: Parallel Speculative Decoding with Adaptive Draft Lengthcited by 85
  7. 2023LLMCad: Fast and Scalable On-device Large Language Model Inferencecited by 45
  8. 2025DSSD: Efficient Edge-Device LLM Deployment and Collaborative Inference via Distributed Split Speculative Decodingcited by 15
  9. 2024PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculationcited by 14
  10. 2024AMUSD: Asynchronous Multi-Device Speculative Decoding for LLM Accelerationcited by 14
  11. 2025SpecEdge: Scalable Edge-Assisted Serving Framework for Interactive LLMscited by 10
  12. 2025Quantize-Sample-and-Verify: LLM Acceleration via Adaptive Edge-Cloud Speculative Decodingcited by 4
  13. 2025Communication-Efficient Collaborative LLM Inference via Distributed Speculative Decodingcited by 4
  14. 2025Fast Collaborative Inference via Distributed Speculative Decodingcited by 4
  15. 2026WISP: Waste- and Interference-Suppressed Distributed Speculative LLM Serving at the Edge via Dynamic Drafting and SLO-Aware Batchingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026AceSpec: An Asymmetric Edge-Cloud Collaborative Framework for Communication-Efficient LLM Inference
  2. 2025DSD: A Distributed Speculative Decoding Solution for Edge-Cloud Agile Large Model Serving
  3. 2025Fast and Cost-effective Speculative Edge-Cloud Decoding with Early Exits
  4. 2026A Pipelined Collaborative Speculative Decoding Framework for Efficient Edge-Cloud LLM Inference
  5. 2026Delay-Adaptive Speculation Control for Low-Latency Edge-Cloud LLM Inference
  6. 2026SPADE: Speculative Decoding for Precise and Low Cost Distributed Edge Cloud Inference
  7. 2026FlexSpec: Frozen Drafts Meet Evolving Targets in Edge-Cloud Collaborative LLM Speculative Decoding
  8. 2026ConfigSpec: Profiling-Based Configuration Selection for Distributed Edge--Cloud Speculative LLM Serving