mlsanityData to 7 Oct 2026

DSD: A Distributed Speculative Decoding Solution for Edge-Cloud Agile Large Model Serving

arXiv 2511.21669 ↗First posted 26 Nov 2025cs.LG cs.DCcited by 5

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

Large language model (LLM) inference often suffers from high decoding latency and limited scalability across heterogeneous edge-cloud environments. Existing speculative decoding (SD) techniques accelerate token generation but remain confined to single-node execution. We propose DSD, a distributed speculative decoding framework that extends SD to multi-device deployments through coordinated draft-target execution. Given the lack of prior work on simulating this paradigm, we first introduce DSD-Sim, a discrete-event simulator that captures network, batching, and scheduling dynamics. Building on insights from DSD-Sim, we further design an Adaptive Window Control (AWC) policy that dynamically adjusts speculation window size to optimize throughput. Experiments across diverse workloads show that DSD achieves up to 1.1x speedup and 9.7% higher throughput over existing SD baselines, enabling agile and scalable LLM serving across edge and cloud.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 21 of its 47 references that are in the map. First posted on arXiv 26 Nov 2025.

Builds on

21 of 47 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges8 Mar

Other references · 20

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2024Qwen2 Technical Reportcited by 2,076
  6. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  7. 2015Teaching Machines to Read and Comprehendcited by 1,039
  8. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  9. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
  10. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  11. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 507
  12. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 318
8 more
  1. 2023Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender Systemcited by 241
  2. 2023SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefillscited by 187
  3. 2017Distributed Deep Neural Networks over the Cloud, the Edge and End Devicescited by 119
  4. 2024Vidur: A Large-Scale Simulation Framework For LLM Inferencecited by 115
  5. 2024PEARL: Parallel Speculative Decoding with Adaptive Draft Lengthcited by 83
  6. 2020CoEdge: Cooperative DNN Inference with Adaptive Workload Partitioning over Heterogeneous Edge Devicescited by 65
  7. 2024LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scalecited by 30
  8. 2022A Multi-agent Reinforcement Learning Approach for Efficient Client Selection in Federated Learningcited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026A Pipelined Collaborative Speculative Decoding Framework for Efficient Edge-Cloud LLM Inferencecited by 3
  2. 2026WISP: Waste- and Interference-Suppressed Distributed Speculative LLM Serving at the Edge via Dynamic Drafting and SLO-Aware Batchingcited by 1
  3. 2026WISV: Wireless-Informed Semantic Verification for Distributed Speculative Decoding in Device-Edge LLM Inferencecited by 1
  4. 2026Speculation at a Distance: Where Edge-Cloud Speculative Decoding Actually Pays Offcited by 1
  5. 2026AceSpec: An Asymmetric Edge-Cloud Collaborative Framework for Communication-Efficient LLM Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Speculative Decoding in Decentralized LLM Inference: Turning Communication Latency into Computation Throughput
  2. 2025SLED: A Speculative LLM Decoding Framework for Efficient Edge Serving
  3. 2026SPADE: Speculative Decoding for Precise and Low Cost Distributed Edge Cloud Inference
  4. 2026ConfigSpec: Profiling-Based Configuration Selection for Distributed Edge--Cloud Speculative LLM Serving
  5. 2026FlexSpec: Frozen Drafts Meet Evolving Targets in Edge-Cloud Collaborative LLM Speculative Decoding
  6. 2025Fast and Cost-effective Speculative Edge-Cloud Decoding with Early Exits
  7. 2025FlowSpec: Continuous Pipelined Speculative Decoding for Efficient Distributed LLM Inference
  8. 2026SPECTRE: Hybrid Ordinary-Parallel Speculative Serving for Resource-Efficient LLM Inference