# DSSD: Efficient Edge-Device LLM Deployment and Collaborative Inference via Distributed Split Speculative Decoding

arXiv: https://arxiv.org/abs/2507.12000 · HTML: https://mlsanity.com/paper/2507.12000/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2025.

## Built on by (most cited first, 15 of 15)

- [FlexSpec: Frozen Drafts Meet Evolving Targets in Edge-Cloud Collaborative LLM Speculative Decoding](https://mlsanity.com/paper/2601.00644/) (2026)
- [A Pipelined Collaborative Speculative Decoding Framework for Efficient Edge-Cloud LLM Inference](https://mlsanity.com/paper/2603.19133/) (2026)
- [CoVSpec: Efficient Device-Edge Co-Inference for Vision-Language Models via Speculative Decoding](https://mlsanity.com/paper/2605.02218/) (2026)
- [GoodSpeed: Optimizing Fair Goodput with Adaptive Speculative Decoding in Distributed Edge Inference](https://mlsanity.com/paper/2512.09963/) (2025)
- [WISP: Waste- and Interference-Suppressed Distributed Speculative LLM Serving at the Edge via Dynamic Drafting and SLO-Aware Batching](https://mlsanity.com/paper/2601.11652/) (2026)
- [WISV: Wireless-Informed Semantic Verification for Distributed Speculative Decoding in Device-Edge LLM Inference](https://mlsanity.com/paper/2604.17701/) (2026)
- [Speculation at a Distance: Where Edge-Cloud Speculative Decoding Actually Pays Off](https://mlsanity.com/paper/2606.25091/) (2026)
- [Efficient and Privacy Aware Edge Cloud Collaborative Inference for Large Language Models](https://mlsanity.com/paper/2607.13093/) (2026)
- [EcoVLA: Energy-Efficient Device-Edge Co-Inference for Vision-Language-Action Models under Real-Time Constraints](https://mlsanity.com/paper/2608.15502/) (2026)
- [Privacy-Aware Split Inference with Speculative Decoding for Large Language Models over Wide-Area Networks](https://mlsanity.com/paper/2602.16760/) (2026)
- [Networking-Aware Energy Efficiency in Agentic AI Inference: A Survey](https://mlsanity.com/paper/2604.07857/) (2026)
- [SpecFed: Accelerating Federated LLM Inference with Speculative Decoding and Compressed Transmission](https://mlsanity.com/paper/2604.25777/) (2026)
- [AsymSpec: Efficient Cloud-Edge Speculative Decoding over Asymmetric Networks](https://mlsanity.com/paper/2608.04974/) (2026)
- [AceSpec: An Asymmetric Edge-Cloud Collaborative Framework for Communication-Efficient LLM Inference](https://mlsanity.com/paper/2609.02514/) (2026)
- [X-CoSD: Communication-Efficient Cross-Vocabulary Collaborative Speculative Decoding](https://mlsanity.com/paper/2609.09166/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have transformed natural language processing but face critical deployment challenges in device-edge systems due to resource limitations and communication overhead. To address these issues, collaborative frameworks have emerged that combine small language models (SLMs) on devices with LLMs at the edge, using speculative decoding (SD) to improve efficiency. However, existing solutions often trade inference accuracy for latency or suffer from high uplink transmission costs when verifying candidate tokens. In this paper, we propose Distributed Split Speculative Decoding (DSSD), a novel architecture that not only preserves the SLM-LLM split but also partitions the verification phase between the device and edge. In this way, DSSD replaces the uplink transmission of multiple vocabulary distributions with a single downlink transmission, significantly reducing communication latency while maintaining inference quality. Experiments show that our solution outperforms current methods, and codes are at: https://github.com/JasonNing96/DSSD-Efficient-Edge-Computing